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Abstract. The industrial robot.'* principal ad van Lag* over traditional automation is 
programm ability- Robots ean perforin arbitrary sequences of pre- stored motions or 
of motion* computed as functions of sensory input, This paper reviews requirements 

for and developments in robot programming systems. The key requirements For 
robot programming systems examined in tlie paper are in the areas of sensing, world 
modeling, motion specification t flow of control] and programming support. Existing 
and proposed robot programming systems fail into three broad categories' guiding 
systems in which the user leads a robot through the motions to be performed. 
robot-level programming systems in which the user writes a computer program, 
specifying motion and sensing, and task- level programming systejns in which the 
user specifies operations by their desired effect on objects. A representative sample 
of systems in each, of these categories is surveyed in the paper. 
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1. Introduction 

The key characteristic of robots is versatility; they can be applied \a a large 
variety of tasks without significant re-design. Tins versatility derives from the 
generality of the rohut's physical structure and contrat, but it can be exploited 
only if the robot can be programmed easily- in ™cue cases., the lack of adequate 
programming tools can make some tasks i 2]] possible to perform. In other cases, 
the cost of programming may be a significant fraction of the total cost of an 
application. For these reasons, robot programming systems play a crucial role in 
ro bot dev clopment. Thi 5 paper uu LLiiic s some key requirem ente of robot prograa u mb Lg 
and reviews existing and proposed approaches to rrierl/.ng these requirements. 

1.1, Approaches to robot prngraminiiiR 

'["hi: earliest and most, widespread method of programming robots involves 
manually moving the robot to each desired position, and recording the internal joint 
coordinated- cur responding to that position. In addition, operations such as closing 
the gripper or activating a welding gun are specified at some of these positions. The 
resulting "program" is a sequence of vectors of pint coordinates pins activation 
signals for external L-L|uipnient. Such a program is executed by moving the robot 
through the specified sequence of joint cum-di nates and issuing the indicated signals. 
This method of robnL programming is usually known as teaching by showing]. 
in this paper we will use the less common, but more descriptive, term guiding 
[Grossman 77]. 

Roboi. guiding is a programming encthod which is simple to use and to 
implement. Because guiding can be implemented without a general purpose 
compute^ it was in widespread use for many years before it was cost-effective 
to incorporate computers into industrial robots. Programming by guiding has 
some important limitations,, however, particularly regarding the use of sensors. 
During guiding, the programmer specifies a single execution sequence for the 
robot; there arc no loops, conditionals, or computatioos, This is adequate for some 
applications, such as spot welding, painting, and simple materials handling. !u other 
applications, however, such as mechanical assembly and inspection, one needs to 
specif y the desired action oT the rubot in response to sensory inputs data retrieval, 
nr computation- In these cases, robot program mi rig requires the capabilities of a 
Seneral- purpose computer programming language, 

Some robot system^ provide computer programming languages with command* 
to access sensors and to specify robot motious. We refer to these as explicit or 
robot* level languages. The key advantage of rohoHevel languages is that they 
enable the data from external sensors, such as vision and force, to be used in 
modifying the robot's motions. Through sensing, robots can cope with a greater 
degree of uncertainty in the position of external objects, thereby increasing their 
range of application. The key drawback oF robot- level programming languages, 
relative to guiding, is that they require the roboL programmer to be expert in 
eornputer programming and in the design oF sensor-based motion strategies. Ilcute, 
robotde vcl languages are not accessible to the typical worker on the factory floor. 
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Many recent approaches t« robot programming seek U> provide the power of 
re hot- level languages withuut requiring, programming expertise, One apprnach :? 
to extend the baHie philosophy of guiding to include decision- rnaki rig baser! nn 
sensing. Another approach^ known as task-levd programming, requires specifying 
goals Tor the positions of objects, rather than the motions of the robot needed 
to achieve those goals. In particular, A task- level specification is meant to be 
completely robot- independent;: no positions ar paths that depend on the robot 
geometry or kinematics are specified by the user. Task -level programming systems 
require complete geometric models or the environment and of the robot as input] for 
this reason, they arc abu referred to as u/orld-modchng systems. Neither of these 
approaches is as developed as the guiding and robot-level programming approaches, 
however- 

1,2, Goals of this paper 

The goals of this paper are twofold: one, to identify the requirements, for 
advanced robot programming systems, the other to describe the major approaches 
to the design of these systems. The paper is not meant to be a catalog, of all existing 

robot programming systems. 

A discussion of the require merits for robot programming languages is not 
possible without some notion of what the tasks to be programmed will be and 
who Lhe users are. The next section will discuss one Uwk which is likely Uj be 
representative of robot tasks in the ocar future. We will use this task to motivate 
some or the detailed requirements later in the paper. The range of computer 
sophistication of robot users is large, ranging from factory personnel with zero 
programming experience to PhD's in computer science, [t is a fatal mistake to 
use this fact to argue for reducing the basic functionality of robot programming 
systems to that accession to the; least sophisticated nser. Instead;, we argue that 
robot programming languages should support the functional requirements of its 
iritis 1 , sophisticated i;;:-;-:. i)<\ -;■ i j-n -.: n : i r-- 1 users can vmpkr - -.-- r-i i. gped&J-purftoH 
interfaces,, in thf licmgwatje itself, for the less experienced users. This is the approach 
taken in the design or computer programming languages; it also echoes the design 
principles discussed in [Taylor, Summers, and Meyer £2}, 
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Figure 1- A representative robot application 




2. A robot application 



Figure 1 illustrate!? a representative robot application. The task involves two 
robot? cooperating to assemble a pump. Parts arrive, randomly oriented and in 
arbitrary order, on two moving conveyor belts. The robot sys'-Rm perform a the 
following puncLions: 

Determine the position and orientation oF the parls P using a vision system . 

Grasp the parts on the moving belts- 



1 

2. 
3. 



Place each part on a fixture, add it to the assembly, or put it aside for 

future use, depending on (Jit state of the assembly, 



The following sequence is one segment of the application. The task is to grasp a 
cover on the moving belt, place it on the pump base, and Insert four pins so as to 
al^ri the two parts. Note the central role played by sensory information, 

1. Identify, using visionj the (non- overlapping) parts arriving on one of the 
belts, a pump cover in this case, and determine its position and orientation 
relative to the robotr During this operation, inspect the pump cover for 
defftcta such as missing: holes or broken lah& t 

2. Move ROB0T1 to the pre-speeilieii ^rassp point for the cover h relative to the 

coyePs position and orientation as determined by the vision system. Note 
that if the belt continues moving during the operation, the gra&p point 
will need to be updated using measurements of Lhr bclt : s position. 

3. Grasp the cover using a program rner-epeciJied gripping force. 
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4. Test the measured finger opening against Mi*- 1 expected opening at the 
grasp point. If it is not within the expected tolerance, signal an error. 
Tins condition may indicate that the vision system or the control system 
are malfunctioning. 

5. HI ace the cover on the base, by moving to an approach position above 
the base and moving down until a program mer-speclficd upward force is 
detected by the wrist force sensor. During lb* downward motion, rotate 
the hand so as to null out any torques exerted on the. cover because oF 
misalignment of the cover and the base. Release the cover and retard its 
current position for future use. 

6. In parallel with the previous steps, move ROBOTS to acquire an aligning 
pin from the feeder. Bring the pin to a point above the position of the first 
hole in the cover, computed from the known position of the hole relative 
to the cover and the position of the cover recorded above. 

T. Insert the pin. One strategy For this operation requires tilting the pin 
slightly to increase the chance: a of the tip of the pin falling into the hole 
llnoue Tlj 74]. If the pi" docs nut fall into the hole, a spiral search can 
be initiated around that point [Goto 80, Taylor Tfi]. Once the tip of the 
pin is- seated in the hole, the pin is straightened. During this motion,, the 
robot is instructed to push down with a pre- specified force, to push in 
the jf direction (so as to maintain contact with the side of the hole), and 
move so as to null out any forces in the x direction (Inoue 7^]. At the end 
of this operation, the pin position l& tested to ascertain that it is within 
tolerance relative to the computed hole position, 

&. In parallel with, the insertion of the pin bv ROBOT 2, KflBQTl fetches another 
pin and proceeds with the insertion when ROB0T2 Is done. This cycle is 
repeated until all the pins are inserted- Appropriate interlocks must be 
maintained between the robots in avoid a collision. 

This application makes use of four types oF sensors; 

1. Direct position sensors. The internal sensors, e.g. potentiometer*, or 
incremental encoders, in the robot joints and in the conveyor belts are 
used to determine the position of the rohot and the belt at any instant of 
time. 

2. Vision sensors. The camera above each belt is used to determine the 
identity and position of parts arriving on the belt and to inspect them. 

3. Finger tovr.k ar.nsom. Sensors in the lingers are used to control the 
magnitude of the gripping forte and to detect the presence or absence of 
objects between the fingers. 

4. Wrist force sensors. The positioning errors in ihe robot, uncertainty in 
part positions, errors in gjasping position, and part tolerances all conspire 
to make it impossible to reliably position parts relative to each other 
accurately enough for tight tolerance assembly, ft is possible, however, to 



use the fbir-Es generated as the assembly progresses to suggest incremental 
motions thai will achieve tin; des-ired final state ; this is known as compliant 
moiian, e.g., [Mason. 81]. 

Most of this application is possible today with commercially available robots and 
vision systems. The exceptions are in the use of sensing. The pin insertion, for 
example,, would be done today with a mechanical compliance device |Whilney 
82) specially designed for this type of operation ^ Techniques for implementing 
compliant motion via force feedback are known, e.g., |Paui 81, RaiherL an,d 
Craig Sl r Shimano 78]; but current force feedback methods arc not as fast or as 
robust as mechanical compliance devices. Current commercial vision systems would 
also impose limitations on the task, e.g., parts must not he touching. Improved 
techniques for vision and compliance are key areas of robotics research. 



3. Requiremeris of robot programming 

The task described above illustrates the major aspects, of sophisticated robot 
programming: sensing, world modeling,, motion spcrilication r and flow of control. 
This section discusses each of these issues acid their impact on robot programming. 

3,1, Sensing 

The vast majority of current industrial robot appli cations arc performed using 
position control alone without significant external sensing, instead, the environment 
is engineered so as to eliminate all significant sources of uncertainty. All parts are 
delivered by Feeders, for cm amp I e, so that their positions will be known accurately 
at program mlng time, Special purpose devices are designed to compensate for 
uncertainty in each grasping or assembly operation, Thifv approach requires large 
investments in design time and special -purpose equipment. Tar each new application. 
Because oF the tnagnitnde of the investment, the rangr of profitable applications is 
limited; because of th<! special- purpose nature uf the equipment, the capability of the 
system to respond to changes in the design of the product ur hi the manufacturing 
method is negligible , Under these conditions, much of the potential versatility of 
robots is wasted, 

Sensing enables roboi* to perforin tasks in the presence of significant 
environmental uncertainties, without special -purpose l,n cling. Sensors can he used to 
identify the position of parts, to inspect parts, to detect errors during manufacturing 
operations,, and to accomodate to unknown surfaces. Sensing places two key 
requirements on robot programming systems, The first requirement is tu provide 
general input and output mechanisms for acquiring sensory data. This requirement 
can be met simply by providing the I/O mechani^jis available in most high-level 
computer programming languages, although this has seldom heen done. The second 
requirement is to provide versatile control mechanisms for using sensory data 
to determine robot motions. This need to specify parameters for sensor-based 
motions and to specify alternate actions based on sensory conditions is the primary 
motivation for using sophisticated robot programming languages. 

Sensors are used Tor different- purposes in robot programs; each purpose has 
a separate impact on the system design. The principal uses of sensing in robot 
programming are: 

1. Initiating and terminating motions. 

2. Choosing among alternative actions. 

3. Obtaining the identity and position of objects and features of objects. 

4. Complying to external con strain U- 

Thc most common use of sensory data in existing systems is to initiate 
and terminate motions. Most robot programming systems provide mechanisms For 
waiting for an cxlernal binary signal before proceeding with execution of a program. 
ThiF capability is used primarily to synchronise robots with other machines. One 
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common at>p]ir:ntioi: or this capability arises when acquiring parte From feeders; the 
robot's graspartg motion it initiated when a light-beam is interrupted by the arrival 
of a new part at the feeder. Another application is that of locating an imprecisely 
known surface by moving towards it and terminating the approach motion when 
a micro- switch is tripped or when the value of a force sensor exceeds a threshold. 
This, typo of motion h V^rr.vn as a yvanlcd move [Will and Grossman 75]. Guarded 
moves can be used to identify points on the edges of an imprecisely located object 
such as a pallet. The contact points can than be used to determine the pallet's 
position relative to the: rnbot and supply offset*; for subsequent pick-up motions. 
Section 4.1.2 illustrates a limited form of this technique available within some 
existing guiding systems. General use of this technique requires computing new 
positions on the basss of stores values; hence it is limited La robot- level languag cs. 

The second major use or sensing is in choosing among alternative actions in 
a program. One example is deciding whether to place an object in a fixture or 
a dispose] bin depending on the result of an inspection test. Another, far more 
•::u;ji:ii:jii. (.■;■. ai i i m I l- ;.]■•.:■>■ a :vw„ tesNng whcthei >. grasp f:r insert action had Uie 
desired effect and deciding whether to take corrective action. This type of error 
checking accounts for the majority of the statements in many robot programs. 
Error checking requires the ability to obtain data from multiple sensors, such as 
visual, force, and position sensors, to perform computations on the data, and to 
make decisions on the results. 

The third major use of sensing m robot systems is in obtaining the identity 
and position of objects or features of objects. For sample in the application 
described earlier, a vision module is used to identify and locate objects arriving 
on conveyor belts. Because vision systems are sizable programs requiring large 
amounts of processing h they often are implemented in separate processors. The 
robot program must be able, in these cases, to interface with the external system 
at the level of symbolic data rather than at the level of "raw" sensory data. Similar 
requirements arise in interfacing to manufacturing data bases which may indicate 
the identity of the objects in different positions of a pallet , for example. From these 
considerations we can conclude that robot programming systems should provide 
general input/output interfaces, not just a few binary or analog channels as is the 
rule in today's robot systems. 

Once the data from a sensor or database module is obtained, some computation 
must be performed on the module's output so as to obtain a target robot position. 
For example, ejrisiting commercial vision systems can be used to compute the 
position of the center of area of an object's outline and the orientation of the line 
that minimizes the second moment. These measurements arc obtained relative to 
the camera's coordinate system. Before the object can be grasped, these data must 
be related to the robot's coordinate system and combined with information about 
the relationship of the desired gTasp point to t:ie measured data (see section 3.3). 
Again, this points out the interplay between the requirements for obtaining sensory 
data and for processing it. 

The fourth mode of sensory interaction, active compliance, is- necessary in 



s.jLu anions requiring continuous motion in response to continuous sensory input. 
Dl-.;?i f - s : i r ■ force. proximity, or vi:;i.;:i ^lisd.i. CL.fi U- use-r: '■:. r:n;il:fy 'in- rubu: : s 
motion so us to maintain or achieve a desired relationship with other objects. 
The force-controlled motions 'to turn a crank, for examptej require that the target 
position c*T the robot from instant Lo instant, be determined from the direction 
and magnitude of the forces acting on the robot hand,. c,g, f [Mason SI, Paul and 
Shimafto Tfi). Other examples are welding on an incompletely known or moving 
snirface., and inserting a peg in a hole when the position uncertainty is greater 
than the clearance between the parts. Compliant motion is an operation specific to 
robotics; it requires special mechanisms in a robot programming system. 

There are several techniques for specifying compliant motions, for a review see 
[Mason S3]. One method models the robot as a spring whose- stiFToessea along each 
of the six motion freedoms can be set jHapiafriKa and Asada T7> Salisbury 80], This 
method ensures that a linear relationship is maintained between the Force which 
is sensed and tlm displacements from a nominal position along each of the motion 
freedoms. A motion specification of this type requires the following information; 

1. A coord i i] ate frame in which the force sensor readings are to be resolved t 
known as the constraint /ra-rtifl. Some common alternatives are: a frame 
attached to the robot hand, a fixed frame in the room , or a frame attached 
to the object being manipulated. 

2. The desired position trajectory of the robot. This specifies the robot's 
nominal position, as & function of tune. 

?.. Stiffnesses for each or the motion freedoms relative to the constraint frame. 
For example, a high stiffness for translation along thin z axis means that 
the robot will allow only smflll deviations From the position specified in the 
trajectory, even if high forces are felt in the a direction. A low stiffness, on 
the other hand, means that a small frjrr.fi ran cause a significant deviation 
from the position specified by the trajectory, 

The specification of a compliant motion for inserting a peg in a tide |Mason 83] is 
as follows: 

The constraint frame will he located at the center of the peg's bottom surface r 
with its z-axis aligned 'Aith the axis of the peg. The insertion motion will be a 
linear displacement- in the negative z direction m a position slightly below the 
expected final destination of the peg. Figure 2 illustrates the coordinate system 
and planned insertion motion. 

The HlifTncEKes arc specified by a matrix relating the robot's position parameters 
to the force sensor inputs: 

f=KA 

where f is a fi X 1 vector nf Forces and torques, K is a G X 6 matrix of stiffnesses, and 
A is a G X 1 vector uf deviations of the robot from its plr.nned path- While inserting 
a yeg in a hole h we wish the constraint Trams to follow a trajectory straight down 
the middle of the note, but complying with forces along the x- and y axes and 
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Figure 2. rcg-iikbole insertion illustrating coordinate syasL^m aiul nominal palh.. 
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witfe torques about the i- and jr-axes, Th* stiffness matrix K For this task would 
be a diagonal in atrix 

where ko indicates low stiffness and k\ a high atiflneas'. 

Tiit complexity of specifying the details or a. compliant motion arpies for 
introducing spccial^purpose syntactic mechanisms into robot lan^ua^ea. Several 
such niecl: aftis Km have been prOuused fur difTtrcrciL r.rnn pliant mo Linn types |Mujtaba 
and Goldman 79, Paul and Shimano 7fi, Paul SI, Salisbury BO). 

One kEy difference between the first three sensor interaction mechanisms 
and active compliance is extensibility. The first three methods allow new sensors 
and modules to be added or changed by the user, since the semantics of ihc 
sensor is determined only by the user program. Active compliance, on the other 
J 1 and r requires much more integration between the sensor and the motion control 
subsytcm] a new type of sensor may require a significant system extension. Ideally, 
a user's view of compliant motion could be implemented tr, terms of lower-level 
procedures in the same robot language. Sophisticated users could then modify this 
implementation to suit new applications, new sensors, or new motion algorithms, 
In practice, efficiency considerations have ruled out this possibility since compliant 
motion algorithms must be executed hundreds of times a second 3 , This is not a 
Fundamental restriction, however, and increasing computer power, together with 
sophisticated compilation techniques, may allow future systems to provide this 
desirable capability. 



'Unfortunately, the numerical choices for sliflnre-KE. are dictated by detailed consideration!, of 
charaetefiattcfi of the environment and of the control ay&kma [Wiiitncj ??, Hajtafusit and Aiaxhi 
"] 

? [Gc?tJike Ti\ describes * robot sjalj-m architecture that enable* different aenaor* to bo LnteifneeKi 
inlo the motion control subsystem from, the user Unguage level. He* also |J"aui 31 1 lai a different 
pjoposal. 
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In summary, we have stressed the need for versatile input/output and 
computation mechanisms to support sensing in robot programming systems, The 
most natural approach for providing these capabilities is by adopting a modern 
high-level computer language as the basis for a robot programming language. We 
have identified one sensor-baaed mechanism, namely compliant motion , that requires 
specific language mechanisms beyond those of traditional computer Languages', 

In addition to the direct mechanisms needed to support sensing within robot 
programming languages, there are mechanisms needed due to indirect effects of the 
reliance on sensing far robot programming. Some of these effects arc: 

1- Target positions are not known at programming tame; they may be 
obtained from an external database or vision sensor or simply be defined 
by hitting something. 

2 The actual path to be followed is nut known at programming time; it may 
b(! determined by the history of sensory inputs. 

3- Th-e sequence of motions is not known at programming time; the result of 
sensing operations will determine the actual execution sequence. 

These e (Tects of sensi n g have sign iflcant i in pact on the stru cture of robot program m ing 
systems. The remainder of this section explores these additional requirements. 

3,2. Wurld modeling 

Tasks that do not involve sensing can be specified as a sequence of desired 
robot configurations; there is no need to represeEit the geometrical structure of the 
environment in terms of objects. When the environment is not known a priori, 
however, some mechanism must be provided for representing the positions of objects 
and their features, such as surfaces and holes. Some of these positions are fixed 
throughout the task,, others must be determined from sensory information,, and 
others bear a lixed relationship with respect to variable positions. Grasping an 
object, for example,, requires specifying the desired position of the robot's gripper 
relative to the object's position. At execution time, the actual object position is 
determined using a vision system or on-line database. The desired position for 
the grippcr can be determined by composing the relative grasp position and the 
absolute object position; this grippcr position must then he transformed to a 
robot configuration. A robot programming system should facilitate this type of 
computation on object positions and robot configurations. 

The most common representation for abject positions in robotics and graphics is 
the homogeneous transform, represented by a4x 4 matrix jPaul SI]. A homogeneous 
transform matrix expresses the relation of one coordinate frame to another by 
combining a rotation of the axes and a translation of the origin, Two transforms can 
be composed by multiplying the corresponding matrices. The inverse of a transform 
which relates frame A to frame B is a transform which relates B to A, Coordinate 
frames can be associated with objects and features of interest in a task, including 
the robot gripper or tool. Transfer ms can then be used to express their positions 
with respect to one another. 
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Figure S, World mudci with coordinate frames 
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A simple world model t with indicated coordinate frame5 h is shown in Figure 3, 
The task is S-o visually locjLtn llic bracket on the table, grasp it, and insert- the pin, 
held in a stationary fixture, into the bracket's hole, The meaning of the various, 
transforms indicated in the %ure are as follows. Cam is the transform relating the 
camera frame- to the WOULD frame. Grasp is the transform reiating the desired 
position of the gripper's frame to the bracket's frame. J.et Drackr.1 be the unknown 
transform that relates the bracket frame to WORLD. We will be able to obtain 
from the vision system the value of Bkt, a transform lnlaLing the bracket's IVarnin 
to l.he camera's frame 1 . Hott is a transform relating the hole's frame to that of the 
bracket. The value of Holt is known from the design of the bracket, Pin relates 
the frame of the pin to that of the fixture- Fixture, in turn,, relates the fixture h B 
frame to WORLD. Z relates the fraroe of the robot base to WORLD. Our goat is 
to determine the transform reliLtii]^ the end-effec tor's (gripper's) frame;, E> lebitive 
to the robot's base. Given E acid 2, the robot's joint angles can be determined 
[see, for CKamplc f [Paul fi]]). 

The first step of the task is determining the value of Bracket > which is simply 
Cnrn tikt. '['be desired ^ripper position for grasping the bracket isj 



:| U s-ijiy. o\i]y one camera vrc cannot determine Uv> distance from the camera, to Hi* bracket 
directly. We aaanmc Lnateaid that th? dia-lauca -to lac table ii known. 



2 E — Bracket Grasp 

Since Cam. is relative to WORLD, Bkt relative to Cam t and Grasp relative (o 
Bkt , the composition gives us the desired gripper position relative to WORLD, i.e^ 
Z E. At the target position we want the location of the hole relative to WORLD 
to be equal to that of the pin; this relationship can be expressed as: 

Bracket Hole = Fixture. Pin 
F^rom this we can sec that 

Bracket — Fiii^re fm Hole~ l 
Hence, the new gripper location is: 

Z E — Fixture Fin Hale~* Croup 

The use of coordinate frames to represent positions has two drawbacks. 
One drawback is that a coordinate Frame, in general, does not specify a robot 
configuration uniquely- There may be several robot configurations that place the 
end -effector in a specified frame, For a robot with sis Independent motion freedoms, 
there arc usually on the order of eight robot con figurations to place the gripper at 
a specified frame. Some frames within the robot 1 ? workspace may be reached by 
;m I.--. iff i:innber of conlii'iiriitions, however. Li'ilrthennorC, for robots with more 
than six motion freed orns> the typical coordinate frames m the workspace will be 
achievabie by an infinite number of configurations. The different configuration. 1 ; 
that achieve a frame specification may not be equivalent; Some configu rations-, for 
example, may give rise to a collision while others may not, This indeterminacy needs 
to be sett'ed at programming time, which may be difficult Tor frames determined 
from sensory data. 

Another, dual, drawback or coordinate frames is that- theyTnay over&pecify 
a configuration. When grasping a symmetric object such as a cynndrica] pin, for 
example, it may not be necessary to specify the orientation of the gripper around 
the symmetry axis. A coordinate frame will always specify this orientation, however. 
Thus,, If the vision system describes the pin's position as a coordinate frame and 
the grasping position is specified likewise, the computed grasp position will specify 
ths grifjper's orientation relative to the pin's axis. In some cases this will result 
in a wasted alignment motion; in the worst case, the specified frame may not be 
reachable because of physical limits on joint travel of the robot. Another use of 
partially specified object positions occurs in the interpretation of sensory data. 
When the robot makes contact with an object, it acquires a constraint on the 
position of that object. This in formation does not uniquely specify the object's 
posutloi,, b-iL several such measurements Can be used to update the robot's estimate 
of the object's positions. This type of computation re i[U]res representing partially 
ennstrained positions or, cquivalcnlly, constraints on the position parameters [Taylor 
76, Brooks 81]. 

Despite these drawbacks, coordinate frames are likely to continue being 
the primary representation or positions in robot programs. Therefore, a robot 
programming system should support the representation of coordinate frames and 
computations on frames via transforms. 13ut this is not all; a world model also should 
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Eigure 4. Symbolic spccjlicatiun of positions 




points, are not represented at all, and must still be specified- Therefore, the effective 
uk of CAD databases requires a high-level interface for specifying the desired 

positions. Pointing cm a graphics screen is one possibility, but it suffers from the 
two-dimensional restrictions of graphics [Ambler, Fopplestnne, and KempT R2[. 
Another method [Ambler and Popplestane 75, Popplestone, Ambler* and Bollc-f Kl "'j 
is Lu describe positions hy Beta of symbolic spatial relationships that hold between 
objects in each position. For example, the pn^tinns of fJforfc] in Figure 4 must 
satisfy the following relation ship*: 

(/3 Against fl) and (f4 Against /2) 

One advantage of using symbolic spatial relationships is that, the positions they 
denote are not limited ta the accuracy of a light- pen or of a robot, but that of 
the model- Another advantage of this method is that families of positions smeh as 
those on a surface or along; an edge can be expressed, Furthermore, people easily 
understand these relationships.. One small drawback of symbolic relations is that 
the specifications are less concise than specifications of coordinate frames. 

Another potentially important method of acquiring positions is the use of 
vision. For exam pic- P two cameras can simultaneously track a point of light from a 
laser pointer and the system can compute the position of the point by triangulation 
[llasegawa 32]. One disadvantage of this method and oF methods based on CAD 
models Is that there is no guarantee that the specified point can be reached without 
collisions. 

We have focused on the representation of single positions.; this reflects the 
emphasis in current robot, systems, on end- point specification of motions. In many 
applications, this emphasis is misplaced. For example, in arc-welding, grinding, 
glue application, and many other applications, the robot is tailed upon to follow 
a complex path- Currently tht:se paths are specified as a sequence of positions, 
"3' hr r.*.-:<: ki-iUuji discj^cs alternative \:ic:hx\\h of ik:: : .iTiLn£ m-nUoiiH whir.h require 
representing surfaces and volumes, A large repertoire of representational and 
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computational tools is already available in CAD systems and Numerically Controlled 
(NC) machining systems, e.g., [Faux and Pratt 79]. 

In summary, the data manipulated by robot programs is primarily geometric. 
Therefore, robot programming systems have a requirement to provide suitable 
data input, data representation, and computational capabilities for geometric data. 
OF these three, data input Is the most amenable to solutions that exploit the 
capabilities oF robot systems, e.g., the availability of the robot and its sensors. 

3,3 r Mo linn specification 

The most obvious aspect of robot programming is motion specification. The 
solution appears similarly obvious: guiding, But, guiding is sufficient only when 
all the desired positions and motions are known at programming time- We have 
postponed a discussion of motion specifi ratios] until after a discussion of sensing 
tmd modeling to emphasise the broader range of conditions under which robot 
motion must be specified in sensor based applications. 

Heretofore^ we have assumed that a robot motion is specified by its final 
position., be it in absolute coordinates or relative to some object. In many cases, 
this is not su 133 dent; a path Tor tlic robot must alsu be speriJicd. A simple example 
oF this requirement arises when grasping parts: the robot cannot approach the 
grasp point From arbitrary directions; it must typically approach from above or risk 
colliding with the part. Similarly,, wlien bringing the part to add to a sub- assembly, 
the approach path must- be carefully specified. Paths are commonly specified by 
indicating a sEquence of intermediate positions, known as via paints, that the robot 
should traverse between the initial and final positions. 

The shape of the path between via points is chosen from among some basic 
repertoire oF path shapes implemented by the robot control system. Three types of 
paths arc implemented in current systems: uncoordinated joint motions, straight 
lines in the joint coordinate space, and straight lines in cartesian space. Each 
of these repre-"»mts a Hi Heirs nt tradeoff between speed of execution and IL naturar 
behavior. They are each suitable te some applications- more than others. Robot 
systems should support a wide repertoire of such motion regimes. 

One important issue in motion spec ill cation arises due to the n on- uniqueness 
of the mapping from cartesian to joint coord i nates. The system must provide 
some welhdefined mechanism for choosing among the alternative solutions. In some 
cases, the user needs to identify which sol\jtu>:i ,x- called For. VAL provides a set 
of configuration commands that allow the user to choose one of the up to eight 
joint solutions available at some cartesian positions. This mechanism is useFul, 
but limited. In particular, it cannot be extended to redundant robots with infinite 
families of solutions or to specify the behavior at a kinematic singularity. 

Some applications, such as arc- welding or spray- painting, can require very 
line control of the robot's speed along a path, as well as of the shape of the 
path |Hrady S3, Paid -Si]- This type or specification is supported hy providing 
explicit trajectory control commands in the programming system. One simple set 



of commands: could specify speed and acceleration hounds on the trajectory, ftl. 
provides Tor additional specifications such as the total time of ths trajectory. Given 
a wide range of constraints, it is very likely that the set of constraints for particular 
trajectories will be inconsistent* The programming system should either provide a 
well-defined semantics for treating inconsistent constraints' ot make it impossible to 
specify inconsistent constraints:. Trajectory constraints, also should be appli cable to 
trajectories whose path is not known at programming time, for example, compliant 
motion*, 

The choice of via points for a task depends on the geometry of the parts, the 
geometry of the robot, the shape of the paths, the robot follows between positions, 
and the placement of the motion in the robot workspace, When the environment is 
not known completely at programming time, the via points must be specified very 
conservatively, This can result in unnecessarily long motions. 

Ail additional drawback of motions specified by sequences of robot configurations 
is that the via points are chosen, typically, without regards for the dynamics of the 
robot as it moves along the path. If the robot if to go through the via points very 
accurately, the resulting motion may have to be very slow, This is unfortunate, 
since it is unlikely that the programmer meant, the vis points exactly. Some robot 
systems assume thai via points are cot meant exactly unless told otherwise. The 
system then splines the motion between path segments to achieve a fast h smooth 
motion, but one that does not pass through the via [joints |l 5 aul 81]. The trouble 
is that the path is then essentially unconstrained near the via points; furthermore, 
the actual path followed depends on the Hpeed of the [notion. 

A possible remedy for both of these problem is to specify the motion by 
a set of constraints between features of the robot and features of objects In the 
environment- The execution system can then choose the "best" motion thatEat-ieifies 
the.se constraints, or signal an error if no motion is possible. This general capability 
is beyond the state of the art in trajectory planning, but a simple form has been 
implemented. The user specifies a nominal carU'dian pudi lor the robot plus some 
allowed deviation from the path; the trajectory planner then plans a joint space 
trajectory that satisfies the constraints [Taylor 79], , 

Another drawback of traditional motion specification is the awkwardness 

o! specifying complex paths accurately as sequences or positions, More compact 
description of the desired path usually exist. An approach fallowed in MC machining 
is to describe the curve as the intersection of two mathematical surfaces. A recent 
robot language j MCL |McDonnell Douglas SO], has been defined as an extension to 
APT, the standard NC language. The ^oal of MCL is to capitalize on the geometric 
databases and computational tools developed within existing AFT systems far 
specifying robot motions. This approach is particularly attractive for domains, such 
as aircraft manufacture, in which many of the parts are numerically machined. 
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Another very general approach to trajectory specification is via user-supplied 
procedures parametcrisBd by id me. Paul |77,, Si] refers to Litis as functionally 
defined motion- The programming system executes the function to nhLain position 
goals. This method can be used, for eitample, to follow a surface: obtained from 
CAD data* turn a cra:ik, and throw ■olhjtcLa. The limiting factor in this approach 
i-, I'hi' <\n:i-t\ ;■:'. •:,:.]'■ :i llu' r'i::.i'lj'j;j can be -liI _L.t'.-Ll, it: l-X li,; : :. i :ubul, iy::ti:::i<:, ilo 
method exists for executing user procedures at servo rates. 

A special ease of functionally defined motion is motion specified an a. function 
of sensor values. One example is in compliant motion specifications, where some 
degrees of freedom are controlled to satisfy force conditions. Another example is a 
rnoLion dcfincrl relaLive La a moving, cunvevor bclL. Ji-Jti] of these cases are common 
enough that special purpose mechanisms have been provided in programming 
systems. There axe sign] tic ant advantages to having these mechanisms implemented 
using a common basic mechanism. 

In sunimary, the view of motion specification as simply sped Tying a sequence 
of positions or robot configurations is too limiting. Mechanisms for geometric 
specification of curves and functionally defined motion should also be provided. No 
existing systems provide these mechanisms with any generality, however. 

3.4. Flow of control 

In the absence of any form of sensing, a fixed sequence of operaLiotiS ii the only 
possible type of robot program. This model is not powerful enough to encompass 
HEosing, however. In general, the program for a sensor- haseri robot must choose 
among alternative actions on the basis of its internal model of the task and the 
data from its sensors. The task of section 2, for example, may go through a very 
complex sequence of states, because Lhc parts ire arriving in random order and 
because Lhe execution of the various phases of the operation is overlapped. In each 
state, the tail? program must, specify the appropriaLE action for each robot. The 
programming system must provide capabilities for making these control decisions, 

The major sources of information on which control decisions can. be based are: 
sensors, control signals, and the world model. The simplest use of this information 
is. to include a test at fixed places in the program to decide which action should 
be taken next, e.g., If (i < j) the a Signal X else Move to T. One irnporta-tit 
application wficrc this type of control is suitable is error detection and correction. 

Kohot opcraLions arc. subject to large uncertainties in the initial state of the 
world and in Lbe elfcct of the actions. As a result, the bulk of robot programming is 
devoted to error detection and correction , Much of this testing consists of comparing 
the actual result of an operation with Lbe expected results. One common example 
is testing the finger opening after a grasp operation differs from the expected value, 
indicating either that Lhe part is missing or a different part Is there. This type of 
tesL can be easily handled with traditional IF-THEN tests after completion of the 
operation. This lest is so common that robot languages such as VAL and "AVE [Paul 
7T| have made it part of the- semantics of the grasp command. 
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Many robot applications also have other requirements that do nut. fait naturally 
within the scope of the IF-THEN conLrnl structure. Robot programs often must 
interact with people or machines, such as feeders, belts, NC machines, and other 
remits. These external processes are c#ccuti]i& in parallel and asynchronously; 
therefore, it is not possible to predict exactly when events of interest to the robot 
program may occur. In the tank or S-nr Licin 2, for example, the arrival of a part 
within the field of view of one or the cameras calls for immediate action: cither 
tine of the robots must be interrupted so as to acquire the part a or the belt must 
ue stopped until a robot can be interrupted. The previous operations may then be 
resumed. Other -examples occur in detecting collisions or part slippage from the 
fingers; monitor processes can be created to continuously monitor sensors, but they 
must be *hlc interrupt the controlling process and issue robot commands without 
endangering ongoing tasks. 



It is. possible to use the signal lines supported by most robot systems to 
coordinate multiple robots and machines. Fuj example, In the sample task, the 
insertion ni the pins into the pump cover (steps 6 through £ in Section 2) requires 
that ROBOT l and ROBOTS be coordinated so as to minim iic the duration of the 
operation while avoiding interference among the- robots.. U we let Robot! be in 
charge, wc can coordinate the operation using the following signal lines; 



:. OET-PIM?: ROBOTS asks if it k safe to get a new pin. 



2. QK-TO-GET: ROBOT 1 says it is OK. 



3. INSERT? : HCBDT2 asks if it is safe to proceed to insert the pin. 



4. OK- TO- INSERT: ROBOT 1 says it is OK, 



5. DONE: ROBOT 1 says it is all over. 



The basic ope-ratioo oF the control programs could he as follows: 



LuiaiuH'ftr*) 
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Rosen 

■»lt for CDYER-AflRlYETj 

Signal 0K-70-CET 

i ^.= 1 

C*ll PliCE-Co.»sr-tQ-Fl3tfiLre 

hit for INSERT-PIN? 

Signal. 0K-TO- INSERT 

if (i < up) t,Ti«/i do 

Teal! Gst-Piii-1 
1 l= i+l] 
else do 

[Signal DONE 
Got* 2] 
Wait for GET-FIJI? 
if (1 < npJ then do 

[Signal Glf-TO-GET 
i r= i+ll 
Call Insert-Pin- 1 
Goto 1 



JLDHQT2 
3; I J signal MNE Goto 4 
Signal GE.T-PM* 
fait for OH-TO-GET 
Call Get-Pin -2 
Signal INSERT- flli? 
f»it tax gH-TO-IflSEHT 
Call Inaflrt-Pin-B 
Goto 3 



This illustration of how a simple coordination task could be done with only binary 
signals alsn serves to illustrate the limitation-* af the method, 

lr The programs are asymmetric; one robot is Hie master of the operation, 
ir the cover can arrive on cither belt and be retrieved by either robot, 
then either an additional signal line is needed: to indicate which robot 
will be the master or both robot systems must be subordinated to a third 
controller- 
s' If one of the robots finds a defective pin, there, is nn way Tar it to cause 
the other robot to insert an additional pin while it goes U> dispose of the 
defective one. The program must allocate new signal lines for this purpose - 
In general, * large number of signals may be needed. 

3. Because one robot docs not know the position of the other one, it is 
necessary Lu coordinate them on the baeis of very conservative criteria, 
e.g., being engaged in getting a pin or inserting a pin. This will result 
in slow execution unless the tasks are subdivided very finely and testa 
performed at each division, which is cumbersome. 

4, The position of the pump cover and the pin-feeder must be known by each 
process independently. No information obtained during the execution of 
the task by one robot can be used by the other robot; it must discover 
the information indepcndenUy- 

Thc difficulties uutlincd above are the due to limited communication between the 
process. Signal lines are a simplc-j hut limited, method of transferring information 
among the processes. In practice, sophisticated tasks require efficient means for 
coordination and for sharing the world model [incluJing the state of the robots) 

between processes. 
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The issue of coordination between cooperating and competing asynchronous 
processes is one of the inost active research areas in CarnpuLer Science. Many 
language mechanisms have been proposed for process synchronisation, among these 
are' semaphores (Dijkstra 6S|j events, conditional critical regions [lluarc 1972: , 
monitors and queues [Brinch Hansen 75], and comunicating sequential processes 
[lloare 78|. Robot systems should build upon these developments, perhaps by 
n sing a language such as Concurrent- Pascal [Urine h Hansen 75 1 or Ada |Ichbiah 
SO] as a base language, A few wis Ling robot, languages, have adopted some of 
these mechanisms, eg,, AL and TEACH [Ituoff 79, 80|. Even the most sophisticated 
developments in computer languages do not address all the robot coordination 
problemSj however. 

When the interaction among robots is subject to critical real-time constraints , 
the paradigm or nearly independent control with periodic synchronisation is 
inadequate. An example occurs when multiple robots must cooperate physically, 
e.g.j in lifting an object too heavy for a:-.>- one Slight deviations from a pre-planned 
pnsil.i-mi trajecLory would cause one of the robots to bear all the weight;, leading to 
disaster. What is needed, instead, is cooperative control of both robots based on 
the force being exerted on both robots by the load [Ishida H, Mason SI, Nakano 
et aJ- 74]- The programming system should provide a mechanism for specifying the 
behavior of systems enure complex than a single robot. Another example of the need 
of this kind of co ordination is in the programming and control or multi-fingered 
grippera [Salisbury and Craig 82]. 

Tn summary,, existing robot programming systems are basnd on the view or a 
robot system, as a single robot weakly linked to other machines. In practice, many 
machines in eluding sensors, special grippers, feeders, conveyors, and several robots 
may he cooperating during a task- Furthermore, the ioterac Lions between them may 
be highly dynamic, e.g., Lo [pain Lain a force between them.,, or may require extensive 
sharing of information. No existing robot programming system adequately deals 
with all of these interactions- In fact, no existing computer language is adcquaLc 
to deal with this kind of parallelism and real- time constraints, 

3,5. Programming support 

Robot applications do not occur m a vacuum. Robot programs often must access 
external manufacturing data, ask users Tor data ar corrective action, and produce 
statistical reports. These function* are typical of most computer applications 
and are supported by all computer programming systems. Many robot systems 
neglect to support them, however. In principle:, the exercise »l these functions can 
bp separated IVum the s|H:ci citation of the task itself but, in practice, they are 
inLimatcly intertwined. A sophisticated robot programming system must first be a 
soph] a Heated programming system. Again, this requirement can be readily achieved 
by embedding the robot programming system within an existing programming 
system. Alternatively, care must be taken in the design of new robot programming 
systems not to overlook the "mundane" programming functions, 



A similar sj-tuaiion exists with respect to progr ana development. Robot program 
development is often ignored in the design of robot systems and., conseoucntiy, 
complex robot programs can be very difficult to debug. The development of robot 
program^ has several characteristics which merit special treatment: 

1. Robot programs have complex side-euVcts and their execution time is 
usually bngj hence it is not always feasible to re-initialise the program 
upon failure. Robot programming systems should allow programs to be 
modified on-line and immediately re-started. 

2. Sensory in Formation and real-time Interactions are nut usually repeatable. 
One useful debusing tool for sensor-based programs provides the ahility 
to record the sensor outputs, foge-ther with program traces. 

3. Complex geometry and motions are difficult to visualize; simulators can 
play an important role in debugging, for example, see [ITcginbotham f 
Dooner, and Case Y9 f Soroka SG\ Meyer Bl], 

These are not minor considerations, they are central to increased usefulness of 
robot programming aystcms, 

Most existing robot systems are stand- alone „ meant to be used directly by a 
single »ser without the mediation of computers. This design made perfect sense 
when robots were not controlled by general- purpose computers; today it makes 
Little sense, A robot system should support a high-speed command interface to other 
computers. Therefore, if a user wants to develop an alternate interface, he need taut 
be limited by the performance of the robot system's user interface. Qo the other 
hand, the user can take advantage of the control system and kinematics calculations 
in the existing system. This design would also facilitate the coord Oration oF multiple 
robots and [[take sophisticated applications easier to develop. 



4. Survey ol" Robot Programming Systems 

In this section, we survey several existing and proposed robot programming 
systems. 

4-1- Guiding 

All robot programming systems support some form of guiding. The simplest 
form of jgLLicEirLg is uj record a sequence of robot positions that can then be "played 
hack^j we call this basic tjmtfz Jig. la robot- level systems, guiding is used to define 
positions while the sequencing: is specified in a program, 

The differences among basic guiding systems arc (a) in the way the positions 
are specified and [h) the repertoire of motions between positions, The most common 
ways of specifying positions are:: by specifying incremental motions on a teach 
pendant, and by moving the robot through the motions, either directly or via a 

master- slave linkage- 
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The incremental motions specified via ibe leach-pendant can. be interpreted 
as: independent motiun of eacn joint between positions, straight lines in the joint- 
coordinate spaee,. <*r straight I Liu a j:j chitesian space relative to some coordinate 
system, e. g 1;1 the robot's base or the robot's end-eFTer.tor. When using the teach- 
pendant, only a feiv position e are usually recorded, on command from the instructor. 
The path of the robot is then interpolated between these position £ utin^ one of the 
three types of motion listed abuvc. 

When moving the robot tb rough the motions directly > the complete trajectory 
can be recorded as a series of closely spaced positions on a fixed time base. The 
latter method is used primarily in spray-painting, where it is important to duplicate 
the input trajectory precisely. 

The primary advantage of guiding is its immediacy: ■what yoii sec is what 
you get. In many eases., however, it is extremely cumbersome, as when the same 
position {or a simple variation) must be repeated at different points in a task or 
when fine positioning ts needed. Farther pnore, Wc bave indicated repeatedly the 
importance of sensing in robotics and the limitations of guiding in the context 
of sensing. Another important limitation of basic guiding is in expressing control 
structures, which inherently require testing and describing alternate- sequences, 

4.1,1. Extended Guiding 

The limitations of basic guiding with respect to Sensing and control can be 
abated, though not completely abolished, by extensions short of a full programming 
language, For example, one of the most common uses ai Sensors in robot- programs 
is to determine the location of some object to be manipulated, After the object 
is located, fiilbcequent motions are made relative to the object 'a coordinate frame. 
This capability can be accomodated within the guiding paradigm If taught motions 
can be interpreted as relative to some coordinate frame that may be modified 
at execution time. These coordinate frames can be determined, for example, by 
having the robot move until a touch sensor on the end -effector encounters an object. 
This is known as a guarded motion or a search, This capability is part of sumc 
commercial robot system^ e.g., ASEA [A SEA]., Cincinatti Milacron [Holt T7]j and 
IBM JCrossmajl 77, Summers and Cirussman ft2]. This approach conld be extended 
to the case when the coordinate frames are obtained from a vision system. 

Some guiding systems also provide simple control structures. For example, the 
instructions in. the taught sequence are tfjveu numbers. Then, on the basis of tests 
on external or internal binary signals, control can be transferred to different points 
in the taugbt sequence. The ASEA and Chncsnatti Milacron guiding systems, For 
example, both support conditional branching, These systems also support a simple 
form of procedures. The procedures can he used to carry out common operations 
performed at different times in the taught sequence , such as cum man machining 
operations applied to palletized parts. The programmer can exploit these facilities 
to produce more compact programs. These control structure capabilitiea are limited, 
however, primarily becau.se guiding systems do not snport explicit computation. 
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To illustrate the capabilities of extended guiding systemSj we present a simple 
task program med ici Llui A SEA rc>hnt : H guiding system . The task is illust-TJiLfid, in 
Figure 5; it consists of picking a series of parts of different height* from & pallet, 
moving them, to a drilling machine, and placing them on a different pallet. The 
resulting program has the following structure: 
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Fl&£ 0#f indicit*e do pickup 

Beginning at procedure 

Skip lujit in=Lrnct.iaa if ll*£ is on 

Nfllt kino d-r> p-j"j dcwrc 
Pickup operation (a*e belof) 
End of common rid* for pickup 
Pea itioning for first pickup 
Ex ecu "-i= jtfGccdur* 
Positioning for KBcnnd pickup 

Execute procwJur* 
UacMnmg and. put- torn operation 
Hub tick* do plcKup 
End of coimoii cade for put, aOID 
Position far fit ft fait dcurfi 
Execute prnc n riu r d 
Position lor B«C0IUJ pTIt <ir,WT\ 



Note that the MOD operation is used with two meanings; [1) to indicate the 
end of a common section: of the PATTERN, and {2} to indicate where the 
coTnttion section is to be executed. The sequence of instructions exEctcd would be: 
10, a, 30, 50, 60, , . . „ LOO, , , . r 130, 30, 40, 170, ... , 200, . . . , 230, 30, 50, , , . 

The key to the pickup operation, is that wc can use a search to locate the top 
sin Face oT thfi parL, so wc need not. know the heights, exactly. The pickup sequence 
con Id be programmed as follows (Lingers are assumed closed initially). 



Prdgranifler action 

1. Position robot to PZ- 

2. Position vertically to PI- 

3. Select speed, for notion 
ta Fl . 



Renarka 

P2 it dd top the fiftorteat part. 

PI is above the highest patt, tkift notlc-A 

insyr»S thftt PI is d(r*irtly abnvr P3. 
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&. PflfiiLifin Trertically to P2. 
6. Select spfl^n] to P3 . 
T. Key code for search and. 
vflrtigftl- dpasratlod. 



Pa i nt- to - po 3 lit maLicD Titb fine paciLJan 
coutrol at the end of the notion 
This marks the €nd of the search 

TT.is code indicates tbat the notion that 
folloKB ii: i treir-eli in vartlea.1 diracLlcn. 



a. ptpf 

9. SeL grip GpaBiJlg and 
Ecloct. »ait,io|5 tip*, 

10. GRIPPERS 

11 . Position bo. P3 . 

32. Select time for notion. 



Floe poa itieiUnj 
Spficlfy flngajr opnniiig 

Insert corur&nd to actuate yrippera. 
Grasping position (relative to P2) . 
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13. FTPL Coordinated joint notion, r»l»tiv« to tbe 

pflSlllsia iflif Ult SSArtlt. 

14. Set- grip opening and Specify fingvr closing 
sa]Rrt, lilting ti»e. 

15. GAIPPERS Instil coiuiaivd t£> ICtuaVfr gi-Ljjftrs-. 



The putdown sequence would be programmed in a similar fashion. 
1.1.2- Off-lint quilling 

Traditional guiding requires that the workspaee for the task, alt the tooling, 
and any parts be available during program development. If the task Involves a 
single large or expensive part, such as an airplane., ship, or automobile, it may 
be impractical to wait until a completed part is available before starting the 
programming; this r.tmld Hclay Win complete- manufacturing process. Alternatively, 
the task environment may be in space or underwater. In these cases, a mock up of 
the task may be built, but a more attractive alternative is available when a CAD 
model of the task exists In this, case, the task model together with a robot model 
can be used to define the program by cJJ-tiTie {finding. In this method, the system 
sirrji.ilal.es the ino Lions of Lhe robot in response to a program or to guiding input 
from a Leach pendant. Off-line guiding offers lhc additional advaoLagns of safety and 
versatility. In pjirticular, it is possible Lo experiment with different arrangements 
of the robot relative to the task so as to find one that, for example, minimises task 
execution time [Hcgiiibotham, Dooncr, and Case 79] 

4.2. ltuboldevcl programming 

In section 3. we discussed a number of important functional issues in the design 
of robot programming systems, The design of robot- level languages, by virtue 
of its heritage in the design of computer languages, has inherited many of the 

controversies of that notoriously controversial field, A few or these controversial 
issues are important in robot programming-: 

1. Compiler vs. interpreter. Language systems that compile high-level 
languages into a I uwer- lei? e I language can achieve great efficiency of 
execution as well as early detection of some classes of programming 
errors. Interpreters, on the other hand, provide enhanced interactive 
environments,, including debugging, and are more readily eHtensible. 
These human factors issues have tended to dominate; most robot language 
systems are interpreter based. Performance limitations of interpreters 
hare sometimes interfered with achieving some useful capabilities, such as 
Func tie ri ally defined motions, 

2. New vs. old. Is it better to design a new language or extend art old onef 
A new one can be tailored to the need oT the new domain, An old one 
is likely to be more complcte r lo have an established user group, and to 
have supporting software packages, fn practice, few designers can avpid 
the temptation of starting de novo; therefore most robot languages are 



LHj|>i»u-l l ir*h J in bo I- l'riJiyMiiuiir>R 

■|.i-,v : ifi:it:.'iii^:s 1;ji::ic ;::i-. h: fuhl.L.-Hi. 'J.lliiUl •■?. In i li --: | ■ 1 1 1 \\i£ i- ■ ■ u : <:. t-s 
Tor existing language systems. One advantage of interpreters in this regard 
is that they are smaller programs than compilers and, therefor^ easier to 
build- 
in Lhc remainder of the section, we examine some representative: roboHevel 
programming systems, in roughly chronological order. Trie languages have been 
chosen to span a wide range of approaches to robot- level programming. We use 
examples to illustrate the "sLyIe" q( the languages- a. detailed review of all these 
languages is beyond the scope of this paper. We close the section with a brief 
mention of some of the many other robot- level programming systems that have 
been developed in the past ten yca.ru. 

4.2.1. MHT 1WIMOT1 

The first robot- Level programming language, MH1, was developed for one of 
the earliest computer-controlled robots d the MH-1 at MIT [Ernst 61 1. As opposed 
to its contemporary the Unimatflj which was not controlled by a general-purpose 
computer and used no external sensors, MIT-l was equipped with several binary 
touch sensors throughout its hand, an array of pressure sensors between the 
fingers, and photo-diodes r>n the bottom of the fingers. The availability of sensors 
fundamentaly affected the mode of programming developed for the MH-1. 

UKI [Mechanical Eland Interpreter) ran cm act interpreter implemented on 
the TX-U r.nmputEr. The programming style in MHT was framed primarily around 
guarded cuuVeS, i.e., moving until a sensory condition was detected. The language 
primitives were: 

1. racve: indicates a direction and a speed. 

2. until: test a sensor For some specified condition. 

3. if goto; branch to a program label if some condition is detected. 

4. if cQntJ.nu.tf ; brand] tu continue action if some condition holds. 
A sample program, taken from [Ernst ol^ follows: 

a, now x far 1W ; Hots Along x witb speed 120 

until si ID rfll Lai ; until sense organ 1 

i indicates a decrease or 10, rtlatiifo 

; to the 1,' alii i- at st-art nf this st.flp 

; fgopditifln li 
Until r| 206 kul afas 5tp ; qz until sense organ 1 indicates 

■ iQfi or 1«ef &b*o3iiT9. tben step. 

; (condition 2) 
ifgoto II, l> ; if condition 1 alone is lulfillnd 

; go to saqu*jseft b 
if goto t f2 ; if it i*a«t condtLlon 2 Lb fulfilled 

; f<i :.- "r^.iiM-.-r r 
: f:: :::■:. ::in:- '..,■- | in all otber cag«£ continue flfrqueni* i 
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MH1 did not- support- arithmetic or any other control structure hey/ond sensor 
monitoring. The kngmage, still, is surprisingly ■"modem" and powerful. It was Us 
be many years, before a more genera] language was implemented. 

1.2.2. WAVE 1970-1975 

The WAVE [Paul 77] system., developed at Stanford, was the earliest system 
designed as a general -purpose robot programming Language. WAVE was a "new" 
language, whose syntax was modeled after the assembly language oa the PDP-10. 
tAYE ran off-line as an assembler on a I'DFTU and produced a trajectory file 
which was executed on-line by a dedicated FDP-6. The philosophy in VhVK was 
that motions could be pre-planned and that only small deviations hum these 
motions would happen during execution. This decision was motivated by the 
computation- intensive algqrithms employed by WAVE for LrajcrtDry pi aiming and 
dynamic compensation. Uetter algorithms and faster computers have removed this 
rationale from the design of rohot systems today. 

In spite of WAVE'S low- level syntax, the system provided an extensive repertoire 
of high- level functions. HAVE piojmcrcd several important mechanism? in robot 
programming sy stems \ among these were: 

1. The description of positions by the cartesian coordinates of the end-effector 

(xjjZ, and three Euler angles], 

1. The coordination of joint motions to achieve continuity in velocities and 
accelerations. 

3, The specification of compliance in cartesian coordinates. 

The following program in WAVE, from (Paul 77 1, serves to pick up a pin and insert 
it into a hole; 

trams Pitt . ... : Location et pm 

TRANS HOLE ... ; Location Of Julio 

ASSIGN TRIES £ ; NubfC Ct pickup Bttflwpt-P 

MOVE PIN ; Hoto to PIH. MOVE first moves in ■+£, 

; then to a point Above PIN, then -Z. 

PIQ0JP: 

CUaSE X ; Ftc-fcup J>1B 

SKIPE 2 ; Skip next instruction if Error 2 cecum 

I (Error 5i lingers closed "beyond azg to CLOSE) 
JUMP OK ; Error -did net eeeur r goto OX 

dri!N S ; Errut did occur, fpftB the fiBftira 

CHANCE E, -IpKIL, . ^ N<?ve do»n cue inch 
SQJG TRIES .PICKMP ; Dec Tenant TBIES, if not ungaUT* 

; jump to PICKUP 
IAIT HO PIW ; Print H JdO PIW and wait for curator 

JIMP FICKUP , Try ay.ain P^itn operator Lyp*e HIDCEED 



LirtiiJiij-l'trti H-ul»v4 t'lvflraiiiiiunc 

DHL 

yD'.'Z HOLE ; Uatm Abate hale 

STOP FY, fill. i Slop «[) 50 «. 

CHUiGE Z„-i.NIL r Q.O : Trj to jo dam one Inch 

EKIPE 23 ; Error 33, fiilsd &e et*jp 

."UhtF 3NJHDLE : Errar did H4t occur (pin hit eurf***) 

FREE'l.X.Y ; Proceed ^it-li in^flr* j<;ti by complying 

• r litfa rorc*s along x ji:J y 

SPIN 2,X,T ; tiita c.an.T'\y wit.li torques ahem:, % wid y 

STOP FV h KIL ■ Stop on 5-0 02 . 

GIANGE Z.-2.MIL.0.D ; Uiic the imerUoo 
WHOLE: 

HAIT HO HOLE ; Failed 



NnLfi 1-"h R use of compliance ami guarded moves to achieve robustness in the presence 
of uncertainty and for error recovery, 

IAVE : fi syntax was dillicult, but the language supported a significant- est of 
robot functions, many of which still arc not available in commercial robot systems. 

-1.2.3. MIDI 1972-1976 

MINI [Silver 73|, developed at MIT, was not a °new" language h rather it was an 
extension to an Existing LISP system hy means of a few functions. The functions 
served as an Interface to a real-time process running on * separate machine. LISP has 
little syntax; it is a large collection of procedures with common calling conventions, 
with no distinction between user and system code- The robot control function! of 
UIHI simply expanded the repertoire of Functions available to the 1 JSP programmer. 
Users could expand the basic syntax and semantics of the basic robot interface at 
will, subject to the limitations or the control system, The principal limitation of 
MINI was the fact that the robot joints were controlled independently. The robot 
used with MINI was uartesian, which minimized the drawbacks of uncoordinated 
point-to-point motions. 

The principal attraction of "The Little Robot System'" (Silver 73j Incus 

71] In which MINI ran was the availability of a high -quality 6-degrce-of-frccdom 
force-sensing wrist [luoue tA , Minsky 72| which enabled sensitive force control of 
the robot. Previous force -control svstesns either set the j^ains in the servos to control 
compliance [In one 7l|, or used the error signals In the servos of the electric joint 
motors to estimate the forces at the hand [Paul 72J, In cither case, the resulting 
force sensitivity was on the order of pounds- Kiwi's sensitivity was more than an 
order or magnitude better (approx- 1 C-3,}, 

The basic functions In MINI set position or force goals fur each of the degrees of 
freedom (SETM), reading the position and force sensors (cetm), and waiting for some 
condition to occur (¥AIT)- We will illustrate the use of MINI using a get of aim pie 
procedures developed by Inone fT-l]_ The central piece of a peg-in-hole program 
would be rendered as follows in IvflNte 
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tDEFUH HOVE- ABOVE (P OFFSET) 

: set x,]r h z G Dals ^nJ tilt till they *re mi-.-i-.t--. 

(K= ^-LOCATION PJ) 

<T= (V-LOCATJDIf P)) 

<?= (PLUS <Z-LQCATian P) OFFSET)) 

(WAIT: '(AND (?*) m) (TZ)»1 



(DEFVN IHSEJIT (HOLE) 

(MCVE-AHOVE HOLE 0.2S) 

; dwfiite x target 1 inch bvlc* current position 

(5ETQ ZTARGET (DIFFERENCE (CETW ZPDS) 1.0)) 

: nuve down until a c*Btiet force is net or until 

■ t th« position targgt ii ut. 

(FZ= LANDIMG-fftflCE) 

(IAIT '(OR (?n) tSE(J (GETM ZPQS) ZTARGEim) 

{com ((SEQ (CETM ZPOS) 2TARGETJ. 

: if th* po^itiaai goal las met, i. 4. no nirf*** encountered 

; conply with. lit*nl farces 

(FX= 03 (FT= 0) . 

; and push 4orn uivtil enough resistance is net- 

(FZ= IBEERTI0N-FOHCE) 

(IAIT 'CFSOJ) 
[T ; if ? surfitt iiie euccujiterfl-d 
CEKftDR INSERT)))) 



MINI did not have any of the geometric and control operation* of IJlVE built 
in, but. most of these could easily be irn piemen ted as LISP procedures. The 
primary functional difference between the twu systems lay in the more sophisticated 
trajectory planning facilities of WAVE. The compensating advantage of MINI was 
that- it did not require any pre- planning; the programs could use arbitrary LISP 
computations to decide on motions in response to sensory input. 

-1.2.4, AL I W4 -present 

AL jKinkel, et al. 74 r Mujlaba and Goldman 79] is ao ambitious attempt to 
develop a high-level language tbat provides all the capabilities required fur robot 
programming as well as the programming features of modern high- level languageSj 
such as ALGOL and PASCAL. AL was designed to support robot-level and task-level 
specification. The robot level has been completed aod will be discussed here; the 
task level development will be discussed in section 4.3. 

AL, like A'/iVE and MINI, runs on two machines. Que machine js responsible far 
compiling the AL input into a lower-level language that is interpreted by a real-time 
control machine. An interpreter for the AL language has been completed, as well 
[Hi] i ford 7E)|. AL was designed to provide four major kinds of capabilities: 

J. The manipulation capabilities provided by the HAVE system: cartesian 
spec i tkation of motions-,, trajectory planning, and compliance. 



2. The capabilities of a realtime language: concurrent execution of processes 
synchro ii La at ion, and on- conditions, 

3. The data. and control structures of an ALGOL-] ike language, including data 
types tor geometric calculations, e.g., vector*, rotations, and coordinate 
framei. 

4. Support for world [Modeling, especially the AFFLXMENT mechanism Tor 
modeling attachments between frames including temporary ones such as 
formed by grasping. 

Aa AL program for the peg- in- hole task is: 

SEE IN "insert peg into hole" 

FRAME p*g_bottX)B, p9g grasp, hol« button, bo", a Lap; 

{ The tbara mates frarwB represent actual poeitioTus oT object J»»uarvs, 

not hind pcSitlOBS } 
peg_botiofc * FRAUE(ni.lr&t r V£CTDE(20,3O r 0)*inclses) ; 
h.ole_l»tfcP" * FRJ^(ftilr*t 1 VECTDR(25 1 .35.0}*iQchesli 
{ Crisp 3 Ilg pgsitiop relative to peg bottom } 

grasped + FALSE; 

{ fhfl top af tho hsln is defined to hiv* >. fixed relation to tbe bottom } 
AFFIX hole._top U> hol*_botton RICIDtT 
AT TflAK5(ni'lrot, 3»zJiat*iaoljeG) ! 

OPEN biiind TS r*&,g dluuiu + 1* laches; 

( Initiate Q\w nation t* the pog, no to Lho do all nation fraau- } 

HOVE bairn TO psg bet-ton * peg grasp; 

IHILE NOT grasped A,NE i « trior CO 
BEGIN 'Attempt grasp* 
CLOSE bhand TU o * Inches; 
IF bh>nd < peg diameter/ 2 

THEN HEGIJf "No object in grasp" 

OPEK Lhund TC peg . diaueter t 1 * lBChflS i 

UQVr. bin TD @ - I * incbos ; { © ln.ddCa.t-eS ciltro-At IseaLidlft } 

BB 

ELSE gnsped +■ TRUE; 

1*1+1; 
JDID- 
IF NOT grasp** THEN ABORT* -F»il*d to grsup Un F«g, H >: 

{ Establish a ined relation between arm and peg } 
AFFIX peg__ cotton TO barn RIGIDLY : 

| Sc'.g that *e jidvd the peg hot ton, not barm J 

HOVE p*g_tottqni TO hal«_*op; 

{ test if a hole- is beloir us } 
HOVE barji TO @ - 1 * Seiches 

ON FORCRfrHat) > 10 - ni:nr.«r. DO A&ORTCN* Hgl**) ; 



Luauiuj-ffrei JithoL J'rwa. in* 

{ Eiert do«i**ri force, ihile -comp lj i r.g i>? sid* forces } 
aWE- pog_liottcm to Jiole_bgt.tT.il DIIlECTLr 

WITH FQRCE_7RAI_ - station IN WORLD 

WTTH KUHCEU&at) = -10 * cnne*t 

WITH FQRCE{][3iitJ = ■ ounces 

WITH FDRCE{yliat} = ♦ arnicas 

EL01UT; 

EHD "insert p*g it hole" 

AL is probably the most complete robot pT«gra:innic]g system yet developed; It was 
the first robot language to be a soptiisticaU-d computer language an well as a robot 
control language. AL has been a significant influence on most later robot languages. 

4,2.5- VAL 1975 present 

VAL [Shimano 79, tlnimation fiOj is the robot language used in the industrial 
robots of Unimatioo Inc., especially th* PUMA series. It was designed to provide 
a subset of the capabil iti« of WAVE on a stand -alone mini-computer. VAL is an 
interpreter; improved trajectory calculation methods have enabled it to forego any 
off-line trajectory calculation phase. Tliis lias improved the ease of interaction with 
the language. The basic capabilities of the VAL language are: 

1. Point- to-pointy joint- interpolated, and cartesian motions (in eluding ap- 
proach and deproach motions]; 

2. Specilieation and manipulation of cartesian coordinate Frames, including 
the specification of locaLions relative to arbitrary frames; 

3. Integer variables and arithmetic, conditional branching, and procedures; 

4. Setting and testing binary signal lines and the ability bo monitor these 
lines and execute a procedure when an event h detected. 

YAL's support of serving is. limited to binary signal lines. These lines can be 
used for synchnmi nation £ind also for limited sensory interaction as shown c-arlier. 
VAL'e support of on-line frame computation is limited to composition of constant 
coordinate frames and Fixed translation offsets on existing frames. It does support 
relative rnnl.ion; this, together with the ability lo halt a motion in response to a 
signal, provide* the mechanisms needed for guarded moves . The basic VAL also has 
been extended to interact with an industrial vision system |C31cason and Agin 79] 
by acqoiring the coordinate frame of a part in the field of view. 

As a computer language, VAL is rudimentary; it most resembles the computer 
language BASIC- VAL onEy supports integer variables, not Moating point numbers or 
character strings. VAL does not support arithmetic on position data. VAL does not 
support any kind of data aggregate such as arrays or lists and, although it supports 
procedures., they may not take any arguments. 

A sample VAL program for the peg-in-hole task, is shown below. VAL does not 
support cocnpliapt motion, SO this operation assumes either that the clearance 
between the peg and hole ia greater than the robot J s accuracy or that a passive 



compliance device is mounted on the robot'? end-cffcctor [Whitney S2j. Tliis limits 
ttLe comparisons that can be made to other, marc general h languages. In the 
example, we assume that- a separate processor is monitoring a force sensor and 
communicating with VAL via signal lines. In particular, signal line 3 goes high if the 
2 component of force exceeds a prc-set threshold. 

SETI tries = 2 

RFJJARK If the hand cl^s^s to lesu Ulan 100 na r go to st&terwnt labelled SO. 

10 GRASP tOO. 20 

REMARK tHhflr»Sa« cnnLloue it statement 30. 

C0TO 36 

RFUAIIK Open the f inhere, dlspUe* d*»B »Hai)( »orld 1 tits ind try &£K.la, 

20 OPEN I &0Q 

DRA1 C, 0,-200 

SET I TRIES = TRIES - 1 

IF TRIES C£ THEN 10 

TYPE HD FIN 

STOP 

REMARK Hove 30Q»n above HOLE follo»iivfc a straight lint , 
50 APPRDS HOLE. 3O0 

REMARK Mnnliar signal lice 3 and till proc-edurfl END IT to STOP thfl pi-p.gr an 

HEkARK if ttie cign*l is *cii*at«d during the ne-it notion. 

REHCTI 3. EHDIT 

APFRDS MOLE. 200 

REMARK Pld not reel force, so continue to HOLE. 

uavEe HOLE 

VAL has been designed primarily for operations involving pre-defined robot positions, 
hence its limited support of computation, data structures, and sensing, A new 
version of the system, VAL -2, is under development which incorporates more support 
for computation and communication with external processes, 

4,2,6. AML 1977- present 

AHL |Taylor, Summers, and Meyer S2| is the robot language used in IBM's robot 
prod nets. AML, like ALj is an attempt at. developing a complete "new" programming 
language for robotics that is also a full-fledged computer language. The design 
philosophy of AML in somewhat diflureiit from that of AL d however. Where AL focuses 
on providing, a rich set of built-in high-level primitives for robot operations, AML ha* 
focused on providing a systems environment whore different user robot programming 
interfaces may he bin It. For example, extended guiding (Summers and Grossman 
S2] and vision interfaces [Lavin and Licherman fcW| can he programmed within the 
AML language itself. This approach is similar to thai followed in MINI. 

AML supports operations on data aggregates, which can he used to implement 
Operations on vectors, rotations, and coordinate frames, although these data types 
are not part- oT the lapgu:£g£. AML also supports joint- space trajectory planning 
subject to position and velocity constraints, absolute and relative motions, and 
sensor monitonne that can interrupt motions, AML does not suDuort cartesian 



motion, compliant motion 7 , affixmerit of frames, or multiple process. An aml 
program for peg-in- hole might be: 

PICKUP: SUBR (PART—DATA, TftlES) e 

MOVE (CR IPPER , M AUETER (PA.KT_ DATA) +0 . 2 J 3 
M0VE(<1,S,3>, XTfZ_P0SITr01^?AFLT_DATA)*^ r (J,i>>; 
TUTf— PnaUPCPART_JMTA. TK1E3) t 
HID; 

Tftlf_PICKWP: SUBR (PART _DAIA, tries) : 
IF TRIES LT 1 THE* HETURlK'HQ PART"}; 
D*roifE(a,-l.Q)i 

IF CRASP (DIAMETER (FART._DATA>) = '1*0 PABT* 
TtLEN mY_FICKUF(PART_DATA, TRIES - 1) : 
EHD: 

GRASP: SUBR (DIAMETER,, F> J 

FMONS: ME* APPLY (t MONITOR, PIM£H_F1HKE£F) ) : 

MDVE-fCFUPFER. 0. FUONS) i 

RETURN ( IF QPDET.TIDN{GMPPER') LE PIAUETES;/2 

THEN 'ND FAJtT' 

ELSE. 'PART' J; 
END; 

INSERT; SUBR (PAE7_. DATA, HOLEh 

FHOHS; NE* APPLTCS HDrilTQfl, TIP_FCTVCE(LAN15INC_F1}1RCE) ) ; 
H0VE(<l r 2 r 3>. HC|_E+<:D.a,.2S*); 
D(KWE.{3, -1.0. FMQNS): 
IF OMDMlTOft(mOWS) - 1 

THEN RETUfUK'Nd HJOLE'}- 
MaVE<3, HQLEU) * PAftT_LENC.TH (PART—DATA)): 
EM; 

PART^IK—HOLE: 5VBH (PART—DATA, HOLE) ; 
<P1CKUP PA11T_D*TA. a,); 
(INSERT PART_DATA HOLEJ | 
END, 

This example has shown the implementation of low- 1 eve] routines such as CR ASP n 
thai are available as primitives in AL and VM. In general, such routines would 
be incorporated into a programming library available to users and would be 
indistinguishable from built :n routines. The important point is that such program!; 
can be written in the language. 

The AML language desig" lias adopted many decisions from the designs of the 
LISP and. APL programming languages- AlfL, like LISP, does not make distinctions 
between system and user programs, Also Mil provides a versatile uniform data 
aggregate, similar to LISP'b lists, whose storage is managed by the system. AML, 
like APL and LISP, provides uniform facilities for manipulating aggregates and for 
mapping operations over the aggregates, 

7 Compliant motion* 11L la-w-apccd nmikl t:« written as user programa. in AML bj- uatofc iti una or 
I/O n^ratiojia. Pot JiirSi- spuud motion*, the r**l time control pfrcew would hare to be extended. 



The langLiigeSj I AVE, MINI, AL, VAL, acifJ MIL arc well within the mold of 
traditional procedural languages, both in syntax and the semantics, of all except a 
few of their operations. The next three language*, we consider have departed from 
the inain-line of computer programming languages in more, significant ways. 

4.2.7. TEACH J 975 - 19T& 

The TEACH language [Kuoff 79^ 80] was developed as part of the PACS system 
at ftendix Corporation. The PACS system addressed two important issues thai have 
received Little attention in other robot programming systems.: the issue of parallel 
execution of multiple tasks with multiple devices, including a variety of sensors; 
and the issue of defining robot- independent programs- In addressing these issues 
TEACH introduced several key innovations; among these were: 

1. Programs are composed of partially ordered sequence* of statements that 
can be executed sequentially or In parallel, 

2. The system supports very flexible mapping between the logical devices, 
e.g., robots and. Fixtures, specified in the program and the physical devices 
that carry them out, 

3. All motions are specified relative to local coordinate frames, bo as to enable 

simple re-location of the motion sequence. 

These features are especially important in the context of systems with multiple 
robots and sensors, which are likely to be common in future applications. Few 
attempts have been made to deal with the organisation and coordination problems 
of complex, tasks with multiple devices, not all of them robots. Knoll' jjSO] reports 
that even the facilities or TEACfl proved inadequate in coping with very complex 
applications and argues for the use of model- hascd programming tools. 

4,2, fc*. PAL 1078 present 

PAL [Takase h Paul, aod Uerg 73] is very different in conception from the 
languages we have considered thus far, PAL programs consist primarily of a sequence 
of homogeneous coordinate equations involving the location & of objects and of the 
robot's end-effector. Some of the transforms in these equations, e.g., those specifying 
the relative location of a feature to an object's frame, are defined explicitly in the 
program. Other coordinate frames are defined implicitly by the equations; leading 
the robot through an executiun of the task establishes relations among these frames, 
KoivLug for the implicitly defined frames completes the program. 

PAL programs manipulate basic coordinate frames that define the position of 
key robot features: Z represent* the base of the robot relative to the world, T6 
represents the end of the sixth (Last) robot link relative to Z, and E represents the 
position of the end-effector tool relative to T6. Motions of the tool with respect to 
the robot base are accomplished by specifying the value of Z + T6 • E, where + 
indicates composjLion of transforms. So, for example, Z + T6 + E = CAM + BKT + 
GRASP specifies that the end-effector should be placed at the grasp position on the 
bracket whose position h known relative to a camera, as discussed in Section 3.2. 



i„aikii&-Y'ii*t Hobo*. J'ro^riiouniii}. 

The MOV <eip> command in PAL indicates that the "generalised" robot too] 
frame, ARM * TOL, is to he moved to *sxp>, For simple motions of the end-effector 
relative to the robot hase, ARM is 1 ■* T6 and TOL is E, We can rewrite ARM to indicate 
that the motion happens relative to another object-, e.g., the example above can be 
rewritten to be 

- BKT - CAM • Z + TG + E - GRASP 
In this case ARM can be set to the transform expression 

- BKT - CAM + Z ■+ T6 
MOV GRASP will then indicate that the end-effector is to be placed on the grasp 
Frame of the bracket, m determined by the camera, Similarly h placing, the pin in 
the bracket's hole can be viewed as redefining the tool frame of the robot to be at- 
the hole- This can be expressed as 

- FIXTURE + Z + T6 + E - GRASP * HOLE = PIN 
By setting ARM to - FIXTURE * Z ♦ TS and TOL to E - GRASP - HOLE, MOV PIN will 
have the desired effect. Of course, the purpose of setting AfiM and TOL is to simplify 
U)4 expression of related motions in the same coordinate, frame. 

PAL is still under development; the system described in [Takase h Paul, and Berg 
T9] deals only with position data obtained from the user rather than the robot. Much 
oF the development of PAL has been devoted to the natural use of guiding to define 
the coordinate frames. Extensions to this systems to deal with sensory information 
are suggested in |E J aul SI], The basic idea is that sensory information serves to 
define the actual value of some coordinate frame in the coordinate equations. 

1.2.9. KCL 19T9 present 

MCL (McDonnell Douglas SO] Is an extension oF the APT language for 
Numerically Controlled machining to encompass robot control , including the 
following capabilities: 

1, data types, e.g., strings, booleans, reals, and frames; 

2. control structures for conditional execution, iterative execution, and 

multi-processing; 

3, real-time input- and output; 

4. vision interfaca, including the ability to define a shape to He located in 
the visual field. 

Extending APT provides some ease of interfacing with existing machining Facilities 
in eluding in terraces tu existing geometric databases, By retaining AFT compatibility, 
MCL can also hope to draw on the existing body of skilled APT part programmers. 
On the other hand, the APT syntax, which was designed nearly 30 years ago f is 
not likely to gain wide acceptance outside of the NC- machining community. 

4.2.10. Additional systcma 

Many other robot language systems arc reported in the literature, among these 
are: 



1. ML |Will and Grossman 75] is a low-level robot language developed at IllM, 
with operations comparable to th&s* of a computer assembly Language. 
The motion commands specified joint motions Tor an (almost) cartesian 
robot. The language provided support for guarded moves by means of 

SENSOR commands that enabled sensor monitors; when a monitor was 
activated by a sensor value outside of the specified range, at) active motions 
were terminated. ML supported two parallel robot tasks, and provided for 
simple synchronization between the task?, 

2. EMILY [Evana, Garnettj arid Grussman 76] was an off-line assembler tor 
the ML language. It raised the syntax of HL to a level coin parable to 
FORTRAN. 

3. MAPLE [Darringer and BLasgen 75] was an interpreted AL-Like language 
alao developed at IBM. The actual manipulation operations were carried 
out by using the capabilities of the ML system described earlier. MAPLE 
never received significant use. 

1. SIGLA [Salmon 7S] P developed at Olivetti for l.h« SK»MA rouols : supports 
a basic set of joint motion instructions, testing of binary signals, and 
conditional tests. It is comparable tu the ML language in syntactic level. 
SIGLA supports pseudo-naiailel execution oF multiple tasks, and some 
simple force-control. 

5. MAL [Gini, et at. TG|, developed at Milan Polytechnic, Italy „ is a BASIC- like 

language for controlling multiple cartesian robots. The language supports 
multiple tasks and task synchreniaation by means of semaphores. 

6. LA3HA-S [Falck and Parent &}'., developed at IRJA, France, is a VAL-like 

language with support Tor on-line computations,, for arrays, and for 
pseudo- parallel execution of tasks, 

7. LM |Latombe and Maaer Sl]j developed at IMAG r Grenoble^ France, i* 

a language that provides most nf the manipulation facilities of AL in 
a mini-computer implementation. Ltf also supports affix men t, but not 
multi- process hitf. LM is being used as the programming language for a 
recently announced industrial robot produced by Scemi, Inc.. 

8. MIL [Franklin and Vanderbrug 82] f developed at AUTOMATED Inc, 
HAIL includes a large subset of PASCAL ; it supports computations on 
a variety of data types, as well as providing hi gb- level program control 
mechanisms- HAIL provides interfaces to binary vision and robot welding 
systems. The language has a flexible way of defining and accessing input 
or output lines, either as single or multiple bit numbers. fiAiL statements 
are translated into an intermediate representation which can hr executed 
ulliriently while enabling interactive debugging. RAIL is syntactically more 
sophisticated than VAL; it is comparable to AML and LM. EiAlL does not 
support mtilti- processing or afiixment. 

This is not a complete list, new languages are being developed every y^ar, but it is 
representative of the state of the art. 
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4,3, Task- level programming 

Robot-level languages describe tasks by carefully specifying the rebel, actions 
needed to carry it out. The goal of iaak-lcvt-.i programming systems (Park 77], 
on the other hand, ]? to enable task specification to be in terms of operations an 
the objects in the task. The peg-in-hole task, for example, would be described as: 
ivsrRT peg IN HOLEj instead &r the sequence of robpl motions needed to accomplish. 
the insertion. 

A task planner transforms the task-level specifications, into robot-level 
specifi cations. To do this transform ation, the task planner must have a description 
of the objects being manipulated, tJie task environment, the robot carrying out 
the task,, the initial state of the environment, and the desired final state. The 
output uf the task planner is a robot- level program to achieve the desired final 
state when executed in the specified initial stale. If the synthesiaed program is to 
reliably achieve its goal, the planner must take advantage of any cap Abilities for 
compliant motion, guarded motion, and error checking- Ilcncc the task planner 
must synthesize a sensor- based robot-level program. 

Task -level programming is still a subject of research; many unsolved problems 
remain, The approach, however, is a natural outgrowth of ongoing research and 
development in CAD/CAM and m artificial intelligence. 

Task planning can be divided into three phases: modeling, task specification, 
and robot program synthesis. These phases are not computation all y independent 
but they provide a convenient conceptual division of the problem, 

1.3.1. World Modeling 

The world model for a task must contain the following information: 

1. geometric descriptions of all objects and robots in the task environment; 

2. physical description of all objects, e.g., mass and inertia; 

3. kinematic descriptions of all linkages; 

4. descriptions of robot characteristics, e.g., joint limits, acceleration bounds, 
and sensor capabilities. 

Models of task states also must include the positions of all objects and linkages in 
the world model. Moreover, the model must specify the uncertainty associated with 
each of the positions. The role that each of these items plays in the synthesis of 
robot programs will be discussed in the remainder of the section- But first, we will 
explore the nature of each of the descriptions and how they may be obtained. 

The geometric description of objects is the principal component of" the world 
model. The major sources of geometric models arc computer-aided design (CAD) 
systems, although computer vision may eventually become a major source of models 
jlirady Ki], There are three major types of commercial CAD systems, differing on 
their representations of solid objects: 

1. line - objects- arc represented by the lines and curves needed to draw 
them, 
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2. surface - objects, are represented, as a set of surfaces, And 

3. solid - objects are represented as combinations of primitive sohds 

Line systems and some surface systems do noL represent all the geometric information 
needed far task planning. A list of edge descriptions, for example, it not sufficient to 
describe a unique polyhedron., e.g.., [Markowsky and Wesley BO], in genera], a solid 
modeling syst-em is required to obtain a complete description, In solid modekrs, 
models are constructed by performing set operations on a few types of prim ill ve 
vol nine*. The objects depicted in. Figure 6, for example, can, he described as the 
mi km of two ho lid cylinders A and B, a solid cube C t and a hollow cylinder D- The 
descriptions of the primitive and compound objects vary greatly among existing 
systems- For surveys of geometric modeling systems, see jBraid 78j Baer, Eastman, 
and Iknrion 79, Rcquicha SO]- 

The legal motions of an. object are constrained by Lhc presence of other objects 
in the environment and the form of the r.nnstraints depend in detail on the shapes 
of the objects. This is the fundamental reason why a task planner needs geometric 
descriptions of objecta. There are additional constraints on motion imposed by 
the kinematic structure of the robot itself. If the robot is turning a crank or 
opening a valve, then the kinematics oT the crank and the valve impose additional 
restrictions, on the robot's motion. The kinematic models provide the task planner 
with the information required to plan robot motions that are consistent with 
external constraints. Examples oT kinematic models and their use in planning robot 
motions can be found in [Mason Blj, 

The bulk of the information in a world model remains unchanged throughout 
the execution of a task. The kinematic descriptions of linkages are an exception, 
however. As a result of the robot's operation , new linkages may be created and old 
linkages destroyed. For examplej inserting a pin into a hole creates a new linkage 
with one rotational and one trans] ational degree of freedom. Similarly, the effect of 



inserting the pin might be to restrict the motion of one plate relative r .o another, 
thus removing one degree of freedom from a previously costing linkage. The task 
planner must be appraised of these changes, either by having the user specify 
linkage changes with each new task state, or by having the planner deduce the new 
linkages from the task state description. 

In planning robot operations, tnany of the physical character Istics or objects 
play important roles. The mass and inertia of parts, for example, will determine 
how fast they can be moved or how much farce can be applied lo them before 
they fall over, Also r the coefficient oT friction between a peg and a hole affects the 
jamming conditions during insertion [see, «.g., [Ohwcvorlole and Roth 81, Whitney 
82]), Hence, the world made) must include a description of these characteristic*. 

The feasible operations of a robot are not sufficiently characterised by its 
geometrical kinematical, and physical descriptions, We have repeatedly stressed 
the importance of a robot's sensing capabilities; touch, force, and vision. For 
task planning purposes, vision allows obtaining the position or an object to some 
specified accuracy, at execution Lime, Force sensing allows performing guarded and 
compliant- motions. Touch information could serve in both capacities, hut its. use 
remains largely unexplored [Harmon B2], In addition to sensing, there are many 
Individual characteristics of robots that must be described in the world model: 
velocity and acceleration bounds, positioning accuracy of each, of the joints, and 
work space bounds,, for example. 

Much of the complexity in a world model arises from modeling the robot, which 
is done once. Geometric, kinematic, and physical model* of other objects must 
he provided Tor each new task, however. The underlying assumption in task- level 
langauges is that this information would have been developed as pari, of the design 
of these objects. IT this assumption does not hold, the modeling effort required for 
a task-level specification, using current modeling methods, might dwarf the effort 
needed to generate a robot- level program to carry out the task. 

1.3.3. Task Specification 

Tasks can be specified to the task planner as a sequence of models of the 
wurld state at several steps during execution of the task. An assembly of several 
parts, for example, might he specified by a sequence of models as each part is 
added to the assembly. Figure 7 illustrates one possible sequence of models for a 
simple task. All of the models in the task specification share the descriptions of 
the robot 1 * environment and of the objects being manipulated; the steps in the 
sequence differ only in the positions of the objects. Hence, a task specification is, 
Lit first approximation, a modi* of the ruboVs. word Uifccthci with B sequence of 
changes in the positions of tlic model components. 

A model state is given by the positions of all the objects in the environment. 
Hence, tasks may be defined, in principle, by sequences, of states of the world 
model. The sequence of mode! states needed to Fully specify a task depends on 
the capabilities, of the task planner. The ultimate task planner might need only 
a description of the initial and final state of the task. This has. been the goal of 
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much of the research on automatic problem solving within artificial intelligence 
[see f e.g,, [Nilsson SO]). These problem solving systems typically do not specify the 
detailed robot motions necessary to achieve an Operation 8 . These systems typically 
produce a plan where the primitive commands are of the form: PICKUP{A) 
and MQVETQ[p) without Specifying the robot path or any sensory nper&tionB. 
In contrast to these systems, task planners need significant information about 
intermediate slates, but they can be expected to produce a much more detailed 
robot program. 

The positions needed to specify a model state are essentially similar to those 
needed to specify positions Lo robot-level syNkems. The npLjon of using the robot to 
specif]/ positions is nut open, however. The oLhcr techniques described in Section 
3.2 are still applicable. The use of symbolic spatial relationships is particularly 
attractive for high-level task specifications. 

We have indicated that model slates are simply sets of positions and task 
specifications are sequences of models. Therefore, given a method such as symbolic 
spatial relationships for specifying positinn^ we should be able to specify tasks. 
This approach has several important J imitations, however. We noted earlier that 
a set of positions may overspecify a state. A typical example (Finkei 7G| of this 
chliiculty arises with symmetric objccLs, for example A round peg in a round hole, 
The specific orientation of the peg around its. axis given in a model is irrelevant to 
Hie task. This problem can be solved by treating the symbolic spatial relationships 
themselves as specifying the sLate, since these relationships can express families of 
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positions. Another, more fundamental limitation t is that geometric and kinematic 
models of an operation's final stats are not always a complete specification or the 
desired operation- One example or this is the need to specify how hard to tighten 
a holt during aft assembly. In genera], a complete description of a task may need 
to include parameters of the operation & used to reach one task state from another. 

The alternative to task specification by & sequence of model states ia specification 
by a sequence of operations. Thus, instead of building a model of an object in 
its desired position, we can describe Liu- operation by which it can be achieved , 
The description should still be cshject- oriented, not robnt-nrknLed;, for example, 
the target torque for tightening a bell, should be specified relative to the bolt and 
riot the robot joints. Operations will also include a goal statement involving spatial 
relationships between objects. The spatial relationships given in ths go-al not only 
specify positions, they aJso indicate the physical relationships between objects that 
should be achieved by the operation. Specifying that two surface are Against each 
other, for example, should produce a compliant motion that moves until the contact 
is actually detected t not a motion to the position where contact is supposed to 
occur. For these reasons, Existing proposals for task- level programming lang^a^es- 
havu adopted an opt ration-centered approach to task specification [Lieberman and 
Wesley 17, Loaano- Peres 76|. 

The task specified as a sequence of model states in Figure 7 can be specified 

by the following symbolic operations^ assuming that the model includes names For 

object* and object features;: 

PLACE BEARIHC1 50 (SHAFT FITS BEAM N<j 1 , WLt) ANC 

(BEARINGl-BUTTOH AGAINST SHAFT- (-IP) 

PLACE SPACER SO (SHAFT F?TE SPACER. HOLE* AN& 

( SPACER. BOTTQH ACAINET BEAHHiCl.TGP) 

PLACE REARIHG2 SO (SHAFT FITS BEARTTO2.HBLE) AMD 

(BEAR WG2. BOTTOM ACAIK5T SPACER. TOP) 

PLACE tASHER SO (SHAFT FITS WASHER EULE) AMD 

{HASHER. BDTTDU AGAJBST BEARINGS . TOP) 

SCRE»-IN NUT ON SHAFT TO (TOK&UE = *Q) 

The first step in the task planning process is transforming the symbolic spatial 
relationships among object features in the SO clauses to equations on the position 
parameters of objects in the model- These equations must then be simplified as far 
as possible to determine the legal ranges of positions of all objects [Ambler and 
Popplestone 75, Popplestone, Ambler, and Bcllos S0 d Taylor 76], The symbolic form. 
of the relationships is used during program synthesis also- 

W(! have mentioned that the actual positions of ohjects at task execution 
time will differ from those in the model] among the principal sources of error are 
part variation, robot position errors, and modeling errors. Robot programs must 
toleraLc some degree of uncertainty if [hey are to be useful, but programs that 
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guarantee success under wcr^t-case error assumptions are difficult to write and slow 
to execute. Hence, the task planter must use expectations on the uncertainty to 
choose motion and sensing strategies that axe efficient and robust jlnoue 74j. If the 
uncertainty is too large to guarantee success, then additional sensory capabilities 
or fixtures may be used to limit the uncertainty [H rooks 82b, Taytor 7fi], For this 
reason, estimated uncertainties are a key pari of tank specification. 

It is not desirable to specify uncertainties numerically far each position t:T 
each state. For rigid objects-, a more attractive alternative is to specify the initial 
uncertainty of each object and use the task planner to update the uncertainty as 
operations are perFormr;!. r ,_ ::r lirikaiics, information on uncertainty at each of the 
joints can be used to estimate the position uncertainty of each of the links and of 
grasped objects [Brooks 81, Taylor 76]. 

4.X3. Robot Program Synthesis 

The synthesis of a robot progjam from a task specification is the crucial 
phase of task planning. The major steps involved in this phase are grasp planning, 
motion planning, and plan checking. The output of the synthesis phase is a 
program composed of &rasp commands, several kinds of motion specifications., 
sensor commands, and error tests. This program is in a robot-level language Far a 
particular robot and is suitable for repeated execution without re- pi aiming. 

Grasping is a key operation in robot programs since it aflcetu all subsequent 
motions. The grasp planner must choose where to grasp objects so that no collisions 
wil] result when grasping or moving them [Laugier 81, Loeano- Peres 76, 81, Mathur 
74, Wingham 77 j. In addition., the grasp planner must choose grasp positions so 
that the grasped objects are stable in the gnppcr [Brady 82, Hanafusa and Asada 
T6, Paul 72] . In particular, the grasp must be able to withstand the forces generated 
during motion and contact with other objects. Furthermorej the grasp operation 
shoold he planned so that it reduces, or at least does not increase, any uncertainty 
in the position of the abject tu be grasped [Mason 82[, 

Once the object is grasped, the task planner must synthesize motions that will 
aehieve thr desired goal of the operation reliably. We have seen that robot programs 
Involve three basic kinds of motions: free, guarded, and compliant. Motions during 
an assembly operation, for example, may Lave up to four sub motions: a guarded 
departure from the current position, a free motion towards the destination position 
of the task step, a guarded approach to contact at the destination t and a compliant 
motion to achieve the £oaJ position. 

During free tnotioEL, the principal goal is to reach the destination without 
collision \ therefore, planning free motions is a problem in obstacle avoid ancc. Many 
obstacle-avoidance algorithms exist but none of them are both general and efficient. 
The type of algorithm that lias received the most attention are those that build an 
explicit description of the constraints on motion and search for connected regions 
satisfying those eon strain Is ; see, c.jr,., [llrookii 82 a, Brooks nod Lozaoo-Pcrcz R2. 
Kimtze and Schill S2, Loaano-Pere-z SI, Loauno- Peres and Wesley 73, Schwartz and 
fihF.rir tfl.. ft2, Udupa 77|. A simple example of this kind of technique is illustrated in 
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Figure 8, A moving polygon A = U^ ™th distinguished point u As must translate 
among obstacle polygons B r This problem is equivalent to the problem in which 
v A trances among transformed objects C\ r Each C^- represents the forbidden 
positions of v A arising because of potential collisions between A* and B y Any curve 
that does not overlap any of the C*,, is a safe path foi A among the B r Extensions 
of this Approach tan be used to plan the paths of cartesian robots |Lo2ano-Ferex 
81, Loaano-Fercit and Wesley 79l. 

Compliant motions are designed to maintain contact among objects even in 
the presence of uncertainty in the location of the object*; see (Mason 83] for a 
review. The basic ides is that the robot can only control its position along the 
tangent to a surface 9 without violating the constraints imposed by the surface. In 
the direction normal to the surface, the robot can only control Forces if it ia to 
guarantee contact with the surface The planning of compliant motions therefore 
requires models that enable one to deduce the directions which require force control 
and those that require position control. This planning is most complicated when 
the robot interacts with other mechanisms [Mason 81]. 

Compliant motions, assume that the robot is already in contact with an object; 
guarded motions arc used to achieve the initial contact with an object [Will and 
Grossman 75]. A guarded motion in the presence of uncertainty, however, does 
not allow die program to determine completely the relative position oF the objects, 
several possibilities may be possible ae a result of the motion (see Figure 9), A 
strategy, composed of compliant motions, guarded motions, and sensing, must be 
synthesized to reliably achieve the specified goal. In particular, for the example in 
Figure 9, the strategy must guarantee that the desired (in at state is achieved no 
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matter which of the possible states actually is reached, jBrooks 82b, Latambc 82, 
Loaano-FcTcE 76, Losiano-PcreE, Taylor, and Mason. 82, Taylor 76]. 

Most of the difficulty in doing motion synthesis stems from the need to operate 
under uncertainty in the positions of the objects and of the robot. These individual 
uncertainties can be modeled and their combined effect on positions computed, 
The requirements for successful completion of task steps, tact be. used to choose 
the strategy for execution, e.g., an insertion with large clearance may be achieved 
by a positioning motion* while out with little clearance might require a guarded 
motion to find the surface followed by a compliant motion [Brooks #3b, Taylor 
TG]. In general, the uncertainty in the position of objects may be too large to 
guarantee that some motion plan iwiJI succeed. En these cases, non-contact sensing 
such as vieion may be used at run- time to reduce the uncertainty. The task planner 
must decide when such information is likely to be useful, given tin at the sensory 
information also will be subject to error. This phase or task planning has been 
d tabbed plan checking it is treated in detail by [Brooks 82b |. 

Task planning, as described above, assumes that the actual state of the world 
will differ from the world model, but only within known bounds. This will not 
always be the case however; objects may he outside the bounds of estimated 
uncertainty, objects may be of the wrong type, or objects may be absent altogether. 
In these cases and many others, the synthesized programs will jiot have the expected 
result; the synthesized program should detect the failure anri cither correct it or 
discontinue the opEfation, Error detection will avoid possible damage to the robot 



and other parts of the environment. Hence j an important part or robot, program 
synthesis should be the inclusion of sensory tests for error detection- Error detection 
and correction in robot programs is a very difficult problem, but one for which very 
little research is available [Brooks 82b, Gini, Gini, and Somalvico SI, Los ano- Peres 
76|. 

4.3.4. Taifc*level systems 

A number of task-level language systems have been proposed, but no complete 
system has been implemented. We saw above that many fundamental problems 
remain un solved in this area; languages have served primarily as a focus of research, 
rather than as usable systems. 

The KOVE-rNSTATTCE .system [Feidman, et a|. 7lj was the first of the task-level 
system proposal s. A subset of this proposal was implemented (Paul 72], namely, a 
program Lhat those stable grasping positions on polyhedra and planned a motion 
to aprcach and move the object. The planning did not involve obstacle avoidance 
(except for the table surface) or the planning of sensory operations. 

The initial definition of AL |Fmkel, ct al. 74] called for the ability to specify 
mndds if] AL and to allow specification of operations in terms of these models. This 
has ijeen the subject of some research TJinford 79, Taylor 76], but the results have 
not. been incorporated, int.n ?,he existing AL system, Some additional work within the 
context, of Stanford's Acronym system [Brooks 81] has dealt with planning grasp 
positions jBioford 79|, but AL has been viewed as the target language rather than 
the user language. 

Taylor |7&] discuRses an approach to the synthesis, of sensor- based AL programs 
from task-level specifications. Taylor's method relies on representing prototypical 
motion strategies for particular tasks as parameterized robot programs, known as 
procedure skeletons. A skeleton has all the motions, error tests h and computations 
needed to carry out a task, but many of the parameters needed to specify motions 
and tests remain to be specified. The applicability or a particular skeleton to a task 
depends on the presence, of certain features in the model and the values of parameters 
such as clearances and uncertainties- Choices among alternative strategies for a 
single operation are made by first computing the values of a set of parameters 
suedfic to the task, such as the magnitude or uncertainty region for the peg in 
peg-in-holc insertion, and then using these parameters Lo choose die ''best", e.g., 
Fastest, strategy. Having chosen a strategy, the planner computes the- additional 
parameters needed to specify the strategy motions, such as grasp positions and 
approach positions. A program is produced by inserting these parameters inte the 
procedure skeleton that implements the chosen strategy. 

The approach to strategy synthesis based on procedure skeletons assumes that 
tiiL-x geometry for common sub-tasks is predictable and can be divided into a 
manageable number of classes each requiring a different- skeleton. This assumption 
is needed because the sequence oF motions in the skeleton will only be consistent 
with a particular class of gen metrics. The assumption does not seem to be true 
in general. As an example, consider the tasks shown in Kigurt: id A program for 
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task A could perhaps be used to accomplish tasks B and G, but it could not be 
guaranteed to work reliably. In particular, the presence of additional surfaces iti 
taski B and C may generate unexpected contacts, leading to failures. This approach 
contrasts to an approach which derives the strategy directly from consideration of 
the l~fl.sk description [Lo^ano-E^ercM, Taylor, and Mason 83 1. I:i ilcJ - ;l:. ■::!.:• I systems. 
bat.lt types of approatn.es are Likely to play a role, 

i"i': I.AMA system was designed at MIT |LoaariQ- Peres T6, Losano-Ferea 
and Winston 77) as a task- level language,, but only partially implemented, LAMA 

formulated the relationship of task specification, obstacle avoid an ce, grasping, 
skfllfl ton-based strategy synthesis, and error detection "within unc system. More 
recent work at MIT has explored issues in task planning in more detail outside of the 
context or any particular system [Brooks 82a. 82b, Ltiiano-PenJB 8.1, LuiiauD-Percit, 
Taylor, and Mason 82, Mason SL, 82 j. 

AuTOPASS, at IBM [Lieberman and Wesley 77], defined the syntax and semantics 
of a task-level language and an approach to lis imyk-metLtation. A subset of the 
most general operation, the PLACE s^tement, was implemented. The major part of 
the implementation effort focused on a method for planning coll is inn -free paths for 
rartesian robots among polyhedral obstacles [Lozano-Perea and Wesley 79, Wesley, 
et al. SO]. 

RAPT ^Popplestone, Ambler^ and Bellos 78| is an implemented system for 
tran&Forming symbolic specifications of geometric goals, together with a program 
which specifies the directions of the motions but out their length, ioto a sequence nf 
end-effector positions, RAPT N s emphasis has been primarily on task specification; it 
does not deal with obstacLe- avoidance, automatic grasping; or sensory operations. 

Some robot- leve) language systems have proposed extensions to allow some 
task- level specifications. Lli-GEfl [Maier 82} is. an implemented extension to LK 
[Latombe and Maaer SlJ which incorporates symbolic specifications of destinations. 
The specification ofRQ&EX |Week and ZuhlkeSl] includes the ability to automatically 
plan collision-free motions and Lo generate programs that use sensory information 
available during execution. A full- blown fiQBEX., including tbese capabilities, has 
not been implemented. 
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The deficiencies of existing methods For gnometric reasoning and sensory 
planning have prevented implementation of a nomplctc la*k-level robot program ming 
system. There la as, however, been significant progress towards solving the basic 
problems in task planning;, see [W&»Tin-Pcre2 83) for a review. 
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5. Discussion and Conclusions 

Existing robot programming systems have focused primarily on the specification 
of sequences of robot configurations. This is only a smalt aspect of robot 
[>Y\:\.,r\i\\:i:.\ng, however The.- f.nr:l.-;-l j : - : i bs I i;-i i of io;j:j1 urogram mine, :s tha'.. of 
specifying robot operations so that they can operate reliably in the present* of 
uncertainty and error. This has long been recognised in research labs,, but until 
very recently baa found little acceptance in industrial situations. Some key reasons 
for this difference in viewpoint are: 

1. ' I'J 1 1- J;vk r.t -r.'inhlr : : .nr: ;■ 11V >-;.\ .-: | y r : firuisors, especial-}' Lbosc already 
integrated into the control and programming systems of a robot. 

3l Existing techniques Tor sensory processing have tended to be slow when 
compared to mechanical means of reducing uncertainty. 

Both of these problems are receiving significant attention today, When they are 
effectively overcome, the need Tor good robot programming tools will be acute. 

The main goal of this paper has been to assess the stale-oF-the-art in robot 
programming compared with the r«flui rem cuts or sophisticated" robot. Lasiks. Our 
conclusion is that ail or the existing" robot systems fall short of meeting the 
requirements we can identify today. 

The crucial problem in the development of robot programming languages is 
our lack of understanding of the basic issues in robot programming. The question 
of what basic set of operations a robot system should support remains unanswered. 
Initially, the only operation available was joint motion. More recently, cartesian 
motion, sensing and, especially., compliance have been recommitted as important 
capabilities for robot systems. In future systems, a whole range of additional 
operations and capabilities are to be expected: 

1. Increasing integration of sensing and mo (ion; More efficient and 
complete implementations of Compliant motions are a key priority. 

2, CompJcIc object models as a source af data for sensor interfaces and 
trajectory planning: Existing partial models of objects are inadequate for 
most sensing tasks; they are also limited as a source oT path constraints, 
Surface and volume models, together with appropriate computational 
tools, should also open the way for more natural and concise robot 
programs, 

3, Versatile trajectory specifications: Current systems over specify trajec- 
tories and ignore dynamic constraints on motion. Furthemore, they severely 
restrict the vocabulary at path shapes available Lo users. A mechanism 
such as functional I y- defined motion can make it easy to increase the 
repertoire of trajectories available to the user, 

4. Coordination of multiple parallel tasks: Current robot systems have 
almost completely ignored this problem, but increasing use of robots with 



Laiar.o -Vim kofcoL Pregi-aramjni 

more than six degrees- of- freedom, grippers with twelve or more degrees- 
of- freed om, multiple special-purpose robots with two or three degrees- 
of-freedom, and multiple sensors will make the need For coordination 
mechanisms more severe. 

5. The I/O, control, and synchronization cupabiiities of general- purpose 
compvttT programming Janguiag'es: A key problem in the development 
of robot languages has been the reluctance,, on the part of users and 
researchers alike, to accept that a robot programming language must be 
a sophisticated computer language. The evidence seems to point to the 
conclusion that a robot language should be a superset of an established 
computer programming language, not a subset. 

These devel opnienis should be matched with continuing efforts at raising the level 
of robot programming towards the task-levo3. By automating many of the routine 
programming fundi cms, to can simplify the program mine, process and thereby 
expand the range of applications available to robot systems. 

One problem thai has plagued robot programming research has been the 
significant "barriers to entry" to experimental research in robot programming. 
Because robot control systems, on available robots are designed to be stand-alone, 
every research group has to re- implement a robot control system from the ground up. 
This is a difficult and expensive operation, It is \c be hoped that commercial robots 
of the Future will be designed with a view towards interfacing to other computers, 
rather than as stand- alone systems. This should greatly stimulate development of 
the sophisticated robot programming systems that we will surely need in the future. 
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